Benton writes, “How big is BloombergGPT? Well, the company says it was trained on a corpus of more than 700 billion tokens (or word fragments). For context, GPT-3, released in 2020, was trained on about 500 billion. (OpenAI has declined to reveal any equivalent number for GPT-4, the successor released last month, citing ‘the competitive landscape.’)
“What’s in all that training data? Of the 700 million-plus tokens, 363 billion are taken from Bloomberg’s own financial data, the sort of information that powers its terminals — ‘the largest domain-specific dataset yet’ constructed, it says. Another 345 billion tokens come from ‘general purpose datasets’ obtained from elsewhere.
“The company-specific data, named FinPile, consists of ‘a range of English financial documents including news, filings, press releases, web-scraped financial documents, and social media drawn from the Bloomberg archives.’ So if you’ve read a Bloomberg Businessweek story in the past few years, it’s in there. So are SEC filings, Bloomberg TV transcripts, Fed data, and ‘other data relevant to the financial markets.'”
Read more here.
The Wall Street Journal is looking for an editor to lead its coverage of logistics…
The Wall Street Journal seeks an enterprising and ambitious reporter to cover the intersection of…
The Wall Street Journal is seeking a reporter in Washington, DC, to chronicle one of…
Reuters has hired Wall Street Journal reporter Anna Hirtenstein. She will start next month. Hirtenstein has…
Caroline Gage, head of the Americas for Bloomberg News, sent the following announcement to staff:…
Forbes senior editor Amy Feldman is now covering health care. She had been covering industrial innovation and…