Ex-Employees Raise Concerns About Safety at OpenAI

Ex-Employees Raise Concerns About Safety at OpenAI

Ex-Employees Raise Concerns About Safety at OpenAI

  • Post category:USA

The tech world’s collective eyebrows rose last week when Ilya Sutskever, the OpenAI co-founder who briefly led a rebellion against Sam Altman, resigned as chief scientist. Some observers downplayed the departure, noting that Sutskever hadn’t been in the office in months and that he appeared to have left on cordial terms.

But contentious comments by another departing executive have raised questions about whether the company, one of the leading developers of artificial intelligence tools, is too lax on safety.

“Safety culture and processes have taken a backseat to shiny products,” Jan Leike, who resigned from OpenAI last week, wrote on the social network X. Along with Sutskever, Leike oversaw the company’s so-called superalignment team, which was tasked with making sure products didn’t become a threat to humanity.

Sutskever said in his departing note that he was confident OpenAI would build artificial general intelligence — A.I. as sophisticated as the human brain — that was “both safe and beneficial” to humanity. But Leike was far more critical:

Over the past few months my team has been sailing against the wind. Sometimes we were struggling for compute and it was getting harder and harder to get this crucial research done.

Leike spoke for many safety-first OpenAI employees, according to Vox. One former worker, Daniel Kokotajlo, told the online publication that “I gradually lost trust in OpenAI leadership and their ability to responsibly handle AGI, so I quit.” (Such concerns were why Sutskever pushed OpenAI’s board to fire Altman as C.E.O. last year, though Sutskever later said he regretted that move.)

Vox reports that such employees have been worried about OpenAI speedily pushing out ever-more-sophisticated technology — and about Altman reportedly raising money from autocratic regimes like Saudi Arabia to build an A.I. chips venture.

Another issue was OpenAI’s policies for departing employees, which included nondisclosure and nondisparagement clauses. According to that language, former workers risked losing any already vested equity if they spoke up.

After uproar on social media, Altman wrote on X that he was “embarrassed” by the clauses. He said the company was removing that language from exit documents, adding that it had never canceled fully vested equity.

Altman and OpenAI’s president, Greg Brockman, sought to allay concerns. “We believe both in delivering on the tremendous upside and working to mitigate the serious risks; we take our role here very seriously and carefully weigh feedback on our actions,” the two wrote on X this weekend.

Any perception of disregard for safety at OpenAI could become a problem. The company’s investors, including Microsoft, clearly sided with Altman over Sutskever last year in supporting his reinstatement as C.E.O.

But the notion that OpenAI was being cavalier with any threats its products might pose could lead to tighter regulation — and create a serious obstacle in the company’s race against rivals.

The death of Iran’s president raises uncertainty in the Middle East. Ebrahim Raisi and the Iranian foreign minister, Hossein Amir Abdollahian, died on Sunday in a helicopter crash. The death of Raisi — known as a hard-liner and a potential successor to the country’s supreme leader — comes weeks after Iran and Israel came close to open conflict and as Tehran faced questions about the state of its nuclear program.

The United States and Europe get closer to using Russian assets to finance Ukraine. Treasury Secretary Janet Yellen told The Times that Western officials were studying ways to help Kyiv, including by issuing a loan backed by profit and interest tied to $300 billion in Russian assets being held in Europe. Discussions for such a move come as the U.S. and Europe looked for ways to bolster embattled Ukrainian forces.

The fraud trial of Ozy’s founder begins on Monday. Carlos Watson, who sought to portray himself as a leading media entrepreneur, faces several counts of defrauding investors by misrepresenting the company’s financial results, funding and audience data. (He is accused of ordering a lieutenant to impersonate an executive to secure an investment.) Watson’s lawyers said he engaged in the same sort of “puffery” that other publishers like BuzzFeed did.

Since last summer, the biggest banks in the United States have waged an expansive campaign — including an “unprecedented” blitz of TV ads — to water down a set of proposed capital rules that they say would crimp lending and harm the economy.

Those efforts are looking successful, suggesting that banks are gaining increased clout with watchdogs. Spearheading that change in dynamics with Washington is JPMorgan Chase’s Jamie Dimon, according to The Wall Street Journal.

What happened: The agencies involved in drafting the so-called “Basel III Endgame” proposals — the Fed, the Federal Deposit Insurance Corporation and the Office of the Comptroller of the Currency — are reportedly close to introducing less-stringent bank capital requirements. The mandate would make large lenders hold an amount of capital that’s far less than a 20 percent buffer floated last summer, The Journal reports, adding that some aspects of the deal are still being negotiated.

The rules would apply to the biggest of the big banks, those with $100 billion or more in assets. Regulators and some progressive lawmakers, including Senator Elizabeth Warren, Democrat of Massachusetts, have argued that a tougher mandate would give banks ample cushion against any financial shocks and help avert another banking crisis.

Dimon has repeatedly challenged that rationale, including in a Bloomberg interview on Friday in Paris. He also has played a big behind-the-scenes role in weakening the measure, according to The Journal, in part by coordinating lobbying efforts.

In one especially effective strategy, Dimon is said to have instructed fellow bankers to go around Michael Barr, the Fed official who was key in drawing up the original plan, The Journal reports. Executives should instead press their case directly with Fed governors and with Jay Powell, the central bank’s chair.

A slimmed-down proposal could be a huge victory for the banks. As of last month, the banks had set aside more than $180 billion to meet beef-up capital standards, The Financial Times calculates. Such reserves could potentially be poured back into other parts of their business, or used to buy back stock if the capital cushion is diminished.

A prominent unionization effort in Alabama ran aground on Friday, raising questions about organized labor’s efforts to expand its representation across the United States. But the movement scored a win in California on Saturday with a resounding victory at Disneyland.

Mickey, Minnie and Goofy are set to unionize. A majority of the roughly 1,700 performers who play famous cartoon characters at Disneyland voted to join the Actors’ Equity Association, which represents more than 51,000 actors nationwide.

The organizers had campaigned for higher wages and improvements in safety and scheduling at Disneyland Resort.

But a failed vote at Mercedes-Benz plants in Alabama may be more consequential. The stakes for that vote were high: Workers at a Volkswagen plant in Tennessee voted last month to join the United Automobile Workers, helping the union crack the largely Southern, nonunion factories it had long sought to organize.

That victory appeared to serve as a wake-up call for anti-union forces. Seemingly taking lessons from Tennessee politicians’ mistakes, Alabama officials embarked on a robust campaign, including Gov. Kay Ivey calling on other Southern governors to condemn the vote.

To add firepower, anti-union campaigners recruited Nick Saban, the immensely popular University of Alabama football coach, to meet with the workers in an effort to dissuade them. (Saban co-owns two Mercedes dealerships.)

Still, union leaders have vowed to fight on in the South. Shawn Fain, the U.A.W. president, said on Friday after the Mercedes vote that workers still wanted unions. “I believe they want justice, and we’re going to continue doing what we can do,” he said.

Some labor experts say the Alabama vote is a temporary setback. “It just means they’ll have less confidence going to the next plant,” Arthur Wheaton, director of labor studies at the Cornell University School of Industrial and Labor Relations, told The Times. “The U.A.W. is in it for the long run.”

Tech and retail earnings, a Group of 7 summit and Fed minutes will be in the spotlight. Here’s what to watch.

Tuesday: Lowe’s, Macy’s and Amer Sports report results. Microsoft’s annual developers conference kicks off with a big focus on how the software giant plans to incorporate generative artificial intelligence into P.C.s.

Wednesday: Investors will be closely watching Nvidia deliver its latest quarterly earnings statement, with expectations sky-high that the A.I. boom is still driving big bottom-line growth.

Elsewhere, the Fed will publish minutes from its May 1 policy meeting, which may provide further clues about the central bank’s outlook on interest rates.

Thursday: G7 finance ministers and central bank governors are to meet in Italy for a three-day conference. High on the agenda is the state of global commerce amid the Washington-Beijing trade fight.

Deals

Policy

  • Beijing plans to investigate imports of a widely used thermoplastic from the European Union, the United States and other areas, in apparent retaliation against Western trade barriers. (FT)

  • Why Gov. Doug Burgum, Republican of North Dakota and a wealthy former businessman, is on Donald Trump’s running-mate shortlist. (WSJ)

Best of the rest

  • Bruce Nordstrom, who helped turn his family’s regional store chain into a department-store giant, died on Saturday. He was 90. (NYT)

  • A close look at Greg Peters, the Netflix co-C.E.O. who made advertising and a crackdown on password sharing big sources of revenue for the streaming giant. (Bloomberg Businessweek)

  • Bots featuring A.I.-created art now account for the majority of revenue shared with artists at DeviantArt, a top online forum. (Slate)

  • “How to Live Forever” (New Yorker)

We’d like your feedback! Please email thoughts and suggestions to dealbook@nytimes.com.

by NYTimes

Related: , , , ,