This summer I joined Millennium Management as an AI engineer intern on the cloud and AI/ML infrastructure team. In ten weeks I built a code-review tool: when someone opens a pull request, a planner reads the change and hands it to a few specialist agents, one for correctness, one for security, one for performance and so on. A final step merges what they found, drops duplicates and likely false alarms, and posts a short, prioritized review on the pull request.
At the end of the summer I was offered an extension, and I stayed for fifteen more weeks. I expected more of the same. It was a different job. The summer was about making the tool work. The fall was about making it something other people depend on. These are the things I learned in the second half.
The tool is the small part
When I started, I thought the work was the agent pipeline. By the fall, most of my time went to everything around it.
A service at a large company lives in several environments: one for development, one for testing that mirrors production exactly, and production itself. I had to request the testing environment, learn why it must match production while development does not, and move every change through all three. I had to decide how the data would be stored. I had to request service accounts and access groups, connect to the company's sign-in system, register the tool where other internal tools could find it, and write deployment settings that asked for the right amount of CPU and memory.
None of this is glamorous, and none of it was in any course I took. But it is the difference between a demo and a product. After doing it once from start to finish, I know what it takes to stand up a new application in a large organization, and I can estimate it honestly next time.
Users report problems in a fixed order
The complaints came in waves, and each wave only started after the last one was fixed.
First it was speed. A review that takes too long gets ignored, because the developer has already moved on. Running the specialists in parallel, and letting the service handle more work at once, brought a typical review down to a couple of minutes.
Then it was accuracy. Once reviews were fast, people read them closely and noticed the comments that were plausible but wrong for their codebase.
Then it was capacity. When enough teams used the tool, dozens of reviews ran at the same moment and new ones waited in line. To a user, a queued review looks exactly like a slow one. That led to scaling the service with demand, and to a question I had never considered: who pays for the machines when a tool built by one team is used by many?
I could not have predicted this order in advance. I only found it by staying long enough to watch it.
Context beat prompt tuning
The largest accuracy gain did not come from a better model or a cleverer prompt. It came from giving the agents what a human reviewer already has: the title and description of the pull request, the commits that led to it, the recent history of the repository, and the ability to search the code, read-only, while reviewing.
A reviewer who sees only the changed lines guesses. One who can check how a function is used elsewhere does not need to.
To know whether a change actually helped, I used a second model as a judge, and first checked that its scores agreed with reviews written by people. Only then did I trust it to compare versions. Accuracy, speed and cost pull against each other, and without measuring all three I would have improved one by quietly spending the others.
Fixing needs stricter rules than finding
Later the tool learned to fix the problems it found. This is where I became careful. An agent that reads code and an agent that changes code should not be the same process with the same permissions. We split them: one worker can only read, a separate one can write, and after a fix the tool checks that tests pass and nothing conflicts before telling the developer the pull request is ready.
I also added a reviewer for a different kind of problem: changes that are correct but break the people who depend on them, such as a changed interface. The right response there is not a bug report. It is a warning early enough for those users to prepare.
Adoption is part of the engineering
I gave a talk to a large part of the engineering organization. It helped, but it is not why the tool spread. It spread because it is installed per repository, so when one team turned it on, people from other teams saw its comments on shared pull requests and asked for it. The best advertisement was the tool working in front of someone on their own code.
Getting there also meant explaining it to people who review tools for a living, and passing their checks. I used to think of that as overhead. Now I think a tool that cannot pass review is not finished.
What the extra fifteen weeks bought
After ten weeks I knew my own project. After twenty-five I had a rough map of how the whole place works: how other teams build and ship, how the team reviews an incident afterwards so it does not happen twice, how outside tools get evaluated, and how the work connects to what the company does.
That map is what I would have had to rebuild if I had left in August and come back a year later. I do not regret staying.
If you are starting an internship, my advice is short. Take your project all the way into production. Talk to the people who use it. Sit in on meetings that are not about your project. And if you are offered more time, take it, because the most useful lessons come after the first launch.
The views here are my own and do not necessarily reflect those of my employer.