Johann was born and raised in Stellenbosch. He went to Paul Roos Gymnasium and the University of Stellenbosch, where he studied business law and economics where he dropped out to pursue a career in business. However, in 2004, the university granted him an honorary doctorate in economics.

In 2008, he was granted a privileged doctorate from Nelson Mandela Metropolitan University. Portrayed as "isolated" by the Financial Times and Barron's, Rupert rarely gives interviews and shuns public events. He's nicknamed "Rupert the Bear".
Johann served his business apprenticeship in New York City, where he worked for Chase Manhattan Bank for two years and Lazard Freres for three years. He then returned to South Africa in 1979 to found and serve as CEO of the Rand Commercial Bank
He served as RMB's CEO until 1984, when RMB and Rand Consolidated Investments merged to form RMB Holdings.

He left the company to join Rembrandt.

In 1988, the Rembrandt Group founded the Swiss luxury goods company Richemont, which in turn acquired Rembrandt's stake in Rothmans. Richemont also owns luxury brands such as Cartier (jewellery); Alfred Dunhill and Sulka (designer clothing); Seeger (leather bags); Piaget, Baume & Mercier and Vacheron Constantin (Swiss watches) and Montblanc (writing instruments).
'
In 1995, Rembrandt and Richemont merged their respective tobacco businesses into what was then the world's fourth largest cigarette maker, Rothmans International.
In 1999, Rothmans International merged with British American Tobacco (BAT), the world's second largest cigarette manufacturer. Before the split, Remgro held 10% of BAT and Richemont 18.6%.

The Rupert family is also deeply involved in the South African wine and liquor industry, owning the L'Ormarins and La Motte wine estates and having a stake in Rupert & Rothschild Vignerons, the winemaking partnership between the Rupert and Rothschild families (at the time of his death due to a car crash in 2001, Rupert's youngest son, Anthonij, was head of Rupert & Rothschild Vignerons.)

The Ruperts also partially control two of South Africa's largest wine merchant houses, Stellenbosch Farmers' Winery (SFW) and Distillers Corporation, who together produce one of every six bottles of wine in South Africa and nearly eighty percent of the country's brandy. These two companies have merged to form Distell Group Limited.
Big changes were made in 2000 when Rembrandt was restructured into two publicly traded holding companies – Remgro and Venfin.

Remgro oversaw Rembrandt’s traditional assets, while Venfin acquired the technologyoriented assets.
Less than a decade later, Remgro and VenFin merged to form Remgro, of which Johann Rupert is still the chairman.
In 2000, Rupert was appointed chairman and CEO of Richemont, and the company’s nonluxuriesrelated activities were spun off into Reinet Investments in 2008.
Today, Rupert is the chairman of Remgro, Richemont and Reinet.