bloop | Evaluating LLMs on COBOL.
Modernising legacy code with AI
Read in full here:
This thread was posted by one of our members via one of our news source trackers.
Saw on twitter last week that somehow, ChatGPT was getting worse in generating code. Why is this so?
2 Likes
Can you find that tweet again? It’s something very interesting, because Github Copilot is also based on GPT-3/4.
1 Like
I don’t think ChatGPT is getting worse, I think from time to time OpenAI servers are overloaded and it has a mechanism in place to give you the first lazy answer it finds. Typically if you start a new chat or if you insist with your questions it gets better. However, depending on the subject, sometime bad answers is all you will get. I compared ChatGPT with LeMistral and for programming they are both pretty close.
2 Likes