AI's coding promise challenged: Focus shifts to realistic AI integration and optimization.

Origin Reddit Post

r/technology

AI Promised Faster Coding. This Study Disagrees

Posted by u/Aralknight•07/22/2025

Top Comments

u/Latakerni21377

I'm doing selenium, we started doing it because we got cursor. Without it, nobody really cared to even write them And (our, idk, first selenium job) getters don't work with the boilerplate y

u/theirongiant74

As is the headline. It seems we can agree it's a horseshit study in both methodology and size.

u/DanielPhermous

> can't have it both ways. Neither can you. "Those" is a gross exaggeration.

u/somekindofdruiddude

I remember the smug Lisp weenies saying something similar about design patterns in the 90s.

u/somahan

people are overstating AI’s capabilities (mainly the AI companies!). It is not good enough to replace coders (at least yet!). It is a great tool for them to use for simple algorithms, code do

u/boom929

And then there's the part where it leaves off brackets and forgets to replace placeholder values with actual code.

u/KubaMcowski

Did you miss the part where I wrote "it's so wasteful" and I'll probably just download some offline tools? Besides that - I agree, it's weird timeline.

u/GiganticCrow

I mean the potential is there for actual humanity improving things, but that's not what is getting the funding.

u/ShadowBannedAugustus

Coverting XML to JSON? You can do that in like 4 lines of code with almost any high level language and a 20 year old PC is good enough to do it in seconds. Instead we use clusters requiring m

u/GiganticCrow

I know some coders who got very excited about the potential generative ai had around chat gpt 3 days, but have said it's rapidly gone to shit since 4.

u/snan101

I think it's way, way more likely that it'll improve to the point where it actually does a good job and "coding" as it is known today disappears entirely

u/ohdog

These studies muddy the water a lot because it depends so much on how you actually use AI and in what domain. The notion that AI assistance slows you down if used properly is completely insan

u/Fancy-Pair

That’s why it’s here to stay

u/theirongiant74

That's the problem when your study only includes 16 participants, can't have it both ways. Either way it's a horseshit study that's been getting reposted multiple times every day for the last

u/PokehFace

I think it depends on what you're trying to "do faster", which the article is a little vague about. I needed to write some Javascript for one thing in work - I did not care to learn JS from s

u/thallazar

Definitely. We use cursor and have lots of cursor rules attached. Probably not as finely tuned as they could be but it really helps to keep any code generation in line with your standards, or

u/theirongiant74

No it doesn't. Half the developers hadn't used the tools before, when they corrected for experience it showed that those with 50+ hours experience with the tools were faster. Stop reposting

u/Acceptable-Surprise5

My personal experience with properly understanding and compartilizing the code which allows me to ask the right context. Co-pilot enterprise has about a 85-90% succesrate in explaining or giv

u/MrPloppyHead

Also I think people get caught up with the chatGPT type thing but AI is being used in many different ways not just the chatbots. so AI in general has the possibility to be more than just help

u/DJbuddahAZ

No there is an official plug in on the unreal store that tidy up the code The closest I've found to suitable is Ludas AI and its not that great , Arua AI is second , but over all there.is ju

u/DanielPhermous

>it showed that those with 50+ hours experience with the tools were faster. "Those"? It was *one developer*. Please don't misrepresent the study.

u/Latakerni21377

AI writes great javadoc As a qa dev, I also appreciate it filling the repetivive gaps of writing getters, naming locators, etc But any code generated (e.g. Asking to write a new test case b

u/gurenkagurenda

How many times do we need the same tiny study of 16 developers reiterated on this sub? Ah yes, let’s see what Time has to add to the conversation. I’m sure that will be especially insightful.

u/littlebrwnrobot

It’s quite good at scientific plotting with matplotlib in my experience.

u/WloveW

But people need to remember this is just a brief blip on the way to an AI which will be easily able to code everything that you need it to do in one shot. Likely in a few months, this artic

u/stevefuzz

Totally agree. I was reading something about how it's going to degrade our technical writing... I was like, lol I hate technical writing. I always use it to document stuff. Coding though it s

u/DanielPhermous

>It seems we can agree it's a horseshit study Is your entire debating technique to misrepresent people? Not only have I never said that, but I have not commented on the study at all, only

u/WAHNFRIEDEN

Bogus study

u/G_Morgan

It seems like a lot of the benefits boil down to shitty half baked replacements for features nobody was using until they were 'AI'

u/GardenPeep

Place the mouse, select text, place hands on keyboard, type in the correction, place the mouse, select the text, hands on keyboard, type in the correction. (Oops - where was I? )

u/stevefuzz

How though? It becomes sentient? That's another can of worms and all jobs as we know it disappear. Then what? People who code know how far off LLMs are from actually coding for production env

u/AwardImmediate720

Bingo. Any developer gaining efficiency by having AI, or any other tool, repeatedly generate the same code is failing as an engineer by not encapsulating that code and making it generic enou

u/uisuru89

I use AI only to generate proper log messages and for variable naming. I am bad at it both. AI is good generating nice log messages and nice variable names.

u/SkankyGhost

Software dev here, I will always stand by my statement that AI slows down a skilled developer. Unless you're doing something SUPER cookie cutter it will be wrong, it's math is wrong, it's cod

u/GiganticCrow

What?

u/FractalChinchilla

VS Code seems work better (even on the same model) than using the web chat UI - for what it's worth. Not brilliantly, but better.

u/mcel595

Copilot has almost replaced editor shortcuts for me, it's really good at autocompleting boiler plate and I hardly have to correct anything. I have yet to see any success with prompting with C

u/thallazar

The key is in knowing when to apply. I think we just exist in a space right now where people are still learning when to use the tool instead of anything else in their toolbox. For instance, g

u/Agoras_song

Where it helps me is in naming variables. I can type up the prompt, it gets the structure right but I find value in not having to think about variable names.

u/Caraes_Naur

The only promise of "AI" is lower payroll obligations.

u/WloveW

Except instead of decades between the car releases there are going to be months.

u/AwardImmediate720

And my experience with copilot's autocomplete is that it's usually wrong and is always worse than IntelliJ's built-in autocomplete. IntelliJ at least bothers to look at the API of the class

u/KubaMcowski

I've tried to use AI for coding and it did work from time to time, but it usually doesn't. Now I use it only for converting formats (e.g. XML to JSON) or formating data in a way I can presen

u/yukeake

I do a lot of work with perl, some of which other folks wrote ages ago. There are times I run into a block of code that looks like incoherent line noise. I've found LLMs to be good at parsi

u/dftba-ftw

IIRC this study took people *not* using any Ai assisted coding tools, gave them one and then measured the difference. That introduces a *huge* confounding factor of learning the tool. I'd

u/Belsekar

I've seen gains mostly with senior developers. Quickly setting up new projects, commenting code but the refined work still needs to be done w/out AI. It's creating problems with some junior

u/RhoOfFeh

Until LLMs stop confidently asserting the false repeatedly, they're only suitable for politics and upper management positions.

u/AwardImmediate720

This is my experience as well. AI is great for solving problems that we already solved ages ago with modern IDEs and is terrible at solving problems that we haven't.

u/Caraes_Naur

The only promise of "AI" is lower payroll obligations.

u/bobsaget824

lol. This at least the 3rd time I’ve seen it posted here.

u/DiplomatikEmunetey

It's great for quickly creating unit tests, refactoring, repetitive tasks, syntax corrections, explanations, references. It may not give you the exact answer, but it will give you "put you on

u/GiganticCrow

By 'blueprints' we talking Unreal? I can barely trust humans to make those in a readable fashion, I dread to think what ghastly horrors ai would spit out. Although I guess it'd at least line

u/Crawgdor

I heard NFTs were the future from the same people said the Metaverse was the future, who now say AI is the future. Forgive my skepticism.

u/ohdog

No, as far as I understand the study randomly assigned tickets to be AI assisted or not, so the developers didn't get to choose. The study itself says: We do not provide evidence that: AI

u/ohdog

I misinterpreted the first part fair enough. It's an n of 16 so it's not meaningful to compare previous experience and not.

u/eating_your_syrup

I usually provide a link to the docs

u/somekindofdruiddude

My experience as a developer is that if I’m typing the same code over and over I need to figure out how to reuse that code. I should only be writing code to solve novel problems. Once a probl

u/AwardImmediate720

Getters? Who writes those anymore? You say "javadoc" so I know you're in Java and if you're not using Lombok you're doing it wrong. Getters, setters, tostring, builders, constructors, you

u/McCool303

You mean to tell me a trained programmer is more efficient than just random generating code until an LLM creates something barely functional?

u/Needariley

Honestly for rapid prototypers and hobbyists and idea makers who want to use less investment to get something going.. it's good. You can fine tune it to think about that. And avoids repetitiv

u/abskee

Yeah, it really shines when I just highlight a big chunk and ask it "What is going on here?" When it's just too convoluted for me to read through, or it's a language I don't know very well, a

u/coconutpiecrust

My experience is very similar with pretty much anything I use the current LLMs for. It’s good for basic stuff, but something more complex still requires massive amounts of my input.

u/ew73

My experience as a developer has been that AI is fantastic at getting the code close enough that I don't have to type the same thing over and over again, but the details are wrong enough that

u/AwardImmediate720

On the debugging end it's basically given us back the quality of search engines we had over a decade ago before enshittification hit in full force.

u/Acceptable-Surprise5

Copilot saves me so much time writing SQL queries it's insane, it's also just generally really good at boilerplate. The benefit of co-pilot atleast the enterprise version is that it links you

u/Crawgdor

So far feeding AI to other AI only causes the computer version of mad cow.

u/DJbuddahAZ

Same. I am in game development, and there isn't a single AI that can do blueprints at all . Sometimes, it works itself in circl3s , I actually spend more time correcting it than not . Im not

u/DanielPhermous

>These studies muddy the water a lot because it depends so much on how you actually use AI and in what domain. The study invited experienced developers to use AI in whatever manner they f

u/impanicking

Same. Even with good prompting and giving it a ton of context I end up being faster and get less errors. Perhaps one day AI can take in the context of an entire codebase and write better code

u/DanielPhermous

"If AI is allowed, developers can use any AI tools or models they choose, **including no AI tooling if they expect it to not be helpful**. If AI is not allowed, no generative AI tooling can b

u/rollingForInitiative

Where it saves the most time for me is either for debugging large amounts of badly formatted or obscure error messages, or for writing short and concise scripts that do something specific.

u/DanielPhermous

> IIRC this study took people not using any Ai assisted coding tools, gave them one and then measured the difference. Nope. "Developers with prior Cursor experience (who use Cursor in th

u/AwardImmediate720

And if you are doing something SUPER cookie-cutter a lot you should have already encapsulated it for re-use. Whether that's making a class or method or even a project template anything that'

u/grahag

AI will ONLY get better. And when AI can share it's breakthroughs with other AI's, we'll see very serious improvements in not just coding, but everything.

u/MrPloppyHead

It is helpful. Its main downside is that it can make shit up that looks plausible but in actual fact is not. i.e. it needs to validate its results for it to be truly useful.

u/IniNew

The prompt thing is what gets me. Seeing some of the prompts that people are using to write the code seems like just as much effort, with less certainty of success, as writing code itself.

u/ChanglingBlake

AI promised nothing. Its self serving creators promised a lot. And anyone with an ounce of tech knowledge knew they were bullshitting the entire time.

u/believe_inlove

Your goalpost for AI is being a multibillion company?

u/jobbing885

I once asked Copilot to extract duplicate code from a test class. Was not able to do it. I use it for snippets and ask questions that are usually on stackoverflow. In some cases its pretty us

u/AwardImmediate720

See and my experience with anything involving an API is that AI is actively harmful because it will just hallucinate an API based on the key words in your prompt instead of actually looking a

u/AlleKeskitason

I've also been promised Jesus, heaven, salvation and Nigerian prince's money and they were all equally full of shit compared to the AI companies. I've managed to make some simple scripts wit

u/eating_your_syrup

This. AI can do a lot of the boring things for me like converting data and its parsers from a to b or doing input checking or whatever. I mostly use it for rubber ducking and asking about po

u/-The_Blazer-

Yeah something I've noticed is that if you know how to use just a handful of simple automated tools, you can get most of the 'advantages' of AI, without the fear of it getting everything shoc

u/WrongUserID

I can only agree with you, even though I am only a hobby coder. I have made quite a few Python scripts with the help of chatgpt and it gets so very close to what I want, but at the same time

u/harry_pee_sachs

I agree with you, and I also use it a lot to help me understand documentation of open source libraries or frameworks that I might not be as familiar with. It's not always helpful with diving

u/GiganticCrow

That ai bubble has to burst soon, right? MBAs are completely delusional as to what they think it will achieve, and reality has to hit eventually.

u/FineInstruction1397

"METR measured the speed of 16 developers working on complex software projects" 16 developers? you cannot really draw any conclusion from 16 devs!

u/somahan

no its that it is able to actually reason and create something new, as soon as it can do that it can create an infinite amount of possibilities.

u/steveisredatw

I’ve not used ai coding agents since I don’t want to use a new IDE. But my experience with using chatgpt, Claude and grok etc is that my productivity has not gone up at all. The time I save b

u/Inside_End3641

Cars in the 1920's bet couldn't hold a candle to the 1950's.

u/gurenkagurenda

Only one participant in the study had more than 50 hours of prior experience with Cursor, and that developer was much faster with AI. In my experience, devs who actually get a lot out of Cur

u/Panduninja

Yeah, same here. AI helps me avoid typing boilerplate stuff, but I still have to fix most of what it spits out. It's like having an eager intern who gets the general idea but misses the detai

u/scoff-law

Same experience here. Id go a step further and say that the time people spend prompting would have a huge impact if it was spent teaching junior engineers, which IMO is mechanically pretty mu

u/Swirls109

But here is the rub. After all the money poured into it, executives were promised massive changes. If it's just helpful, then they failed at their decision. That won't ever be the story. We w

u/mr_stupid_face

It’s all about quality prompt/ context / spec availability to the context and knowing how to use the tools. You are absolutely right on what you mentioned. I have some awesome,productivity

u/Nulligun

You suck at prompts and you will be left in the dust by vibe coders unless you sto your ego and figure out how to use these tools effectively.

Origin Reddit Post

r/technology

AI Promised Faster Coding. This Study Disagrees

Top Comments

Ask AI About This

Create Your Own