Probably the best way to use them now is not to trust 100% the code that they generate.

Might take a couple of years more, but it will get better.

‘Survey shows that when Devs start to use the tools, they realize they are unreliable’.

Assuming the same architecture, it will probably get better… but not because LLMs stop hallucinating. It’ll get better because they’ll get larger context windows, and they’ll start putting the docs for the frameworks and languages into that context window before starting. It’ll get better when they can reliably test their own work before submitting. And it’ll get better when they have better processes around workflow incorporated into their programming system prompts.

These are all low hanging fruit, so I expect this to be rapidly improving in the coming months.

One thing that I have learned is that documenting what is not supported is often just as important as documenting what is supported, but it’s a commonly overlooked task in documentation leading to both human and AI hallucinations. For example, if a language structure only supports && conditions an LLM might hallucinate a solution that utilizes || conditions. Improving the documentation for LLMs will also help humans.