CommunityNews

CommunityNews

Flame: A small language model for spreadsheet formulas

FLAME: A small language model for spreadsheet formulas.
The widespread use of spreadsheet environments by billions of users presents
a unique opportunity for formula-authoring assistance. Although large language
models, such as Codex, can assist in general-purpose languages, they are
expensive to train and challenging to deploy due to their large model sizes (up
to billions of parameters). Moreover, they require hundreds of gigabytes of
training data. We present FLAME, a T5-based model trained on Excel formulas
that leverages domain insights to achieve competitive performance with a
substantially smaller model (60M parameters) and two orders of magnitude less
training data. We curate a training dataset using sketch deduplication,
introduce an Excel-specific formula tokenizer for our model, and use
domain-specific versions of masked span prediction and noisy auto-encoding as
pretraining objectives. We evaluate FLAME on formula repair, formula
auto-completion, and a novel task called syntax reconstruction. FLAME (60M) can
outperform much larger models, such as Codex-Davinci (175B), Codex-Cushman
(12B), and CodeT5 (220M), in 6 out of 10 settings.

Read in full here:

This thread was posted by one of our members via one of our news source trackers.

Where Next?

Popular General Dev topics Top

AstonJ
Apple co-founder Steve Wozniak is suing YouTube for allegedly allowing scammers to use images and videos of him to defraud people. The s...
New
First poster: wolf4earth
It’s official. Your private communications can (and will) be spied on - European Digital Rights (EDRi). On 6 July, the European Parliame...
New
First poster: dpritchett
It’s not what programming languages do, it’s what they shepherd you to. How many of you have listened, read or taken part in a discussio...
New
First poster: bot
Rewrite it in Rust by ridiculousfish · Pull Request #9512 · fish-shell/fish-shell. (Sorry for the meme; also this is obligatory.) I thi...
New
First poster: bot
Hector Martin (@marcan@treehouse.systems). Attached: 1 image For those wondering why the hell we need all this safety system stuff for...
New
First poster: bot
Declarative GNOME configuration with NixOS. I adore tinkering with my machine, trying new tools, extensions, themes, and ideas. When I w...
New
First poster: joeb
50 Shades of Go: Traps, Gotchas, and Common Mistakes for New Golang Devs. Go is a simple and fun language, but, like any other language,...
/go
New
First poster: alvinkatojr
About accelerationism, NRx, and the intersection of technology, religion, and philosophy: an analysis of the essential ideas in the new A...
New
CommunityNews
The French originated the meter in the 1790s as one/ten-millionth of the distance from the equator to the north pole along a meridian thr...
New
New

Other popular topics Top

Devtalk
Hello Devtalk World! Please let us know a little about who you are and where you’re from :nerd_face:
New
ohm
Which, if any, games do you play? On what platform? I just bought (and completed) Minecraft Dungeons for my Nintendo Switch. Other than ...
New
siddhant3030
I’m thinking of buying a monitor that I can rotate to use as a vertical monitor? Also, I want to know if someone is using it for program...
New
New
PragmaticBookshelf
Rust is an exciting new programming language combining the power of C with memory safety, fearless concurrency, and productivity boosters...
New
PragmaticBookshelf
“Finding the Boundaries” Hero’s Journey with Noel Rappin @noelrappin Even when you’re ultimately right about what the future ho...
New
AstonJ
Seems like a lot of people caught it - just wondered whether any of you did? As far as I know I didn’t, but it wouldn’t surprise me if I...
New
New
husaindevelop
Inside our android webview app, we are trying to paste the copied content from another app eg (notes) using navigator.clipboard.readtext ...
New
PragmaticBookshelf
Develop, deploy, and debug BEAM applications using BEAMOps: a new paradigm that focuses on scalability, fault tolerance, and owning each ...
New