Signal
AI is starting to get what any new employee gets: a trial period on the real job, with a supervisor and an end date. Until now it was tested in simulators. Britain's financial regulator has put two groups of firms, Barclays, NatWest and UBS among them, into live tests where their AI systems deal with real customers while the regulator watches and can stop the test at any time. The EU has written the trial into law, with a plan filed in advance, a limit of six months and every decision reversible. In South Korea 45 AI services run live at financial firms under special approvals, and since April 2026 a minor change of AI model needs only written confirmation instead of a new approval. Companies are doing it unasked. Andon Labs, whose simulated vending-machine test had become too easy for the newest models, leased a real shop in San Francisco and a real café in Stockholm and handed each to an AI. The café's AI ordered 120 eggs for a kitchen with no stove. Anthropic ran an earlier version of the experiment in its own office and concluded it would not hire its own model. A simulation and real life give different answers. Britain's medicines regulator found that a safety control which worked in virtual tests made no difference on real consultations, because real users did not behave like the test. The open question is who decides the trial is over. In Utah an AI renews prescriptions for real patients in a state pilot that plans to reduce the doctor's review in three phases, and the medical licensing board asked for it to be suspended and was refused. A trial period always ends with a person making a decision. For AI, nobody has yet agreed who that person is.
So what?
Doing well in a simulator is becoming the interview. The trial period on real work decides whether an AI system gets the job.
Evidence
Financial Conduct Authority, 21 April 2026 (policy): Britain's financial regulator named eight more firms, Barclays, Experian and UBS among them, to test AI applications live, after a first group in December 2025 that included NatWest, Monzo and Santander. Its terms describe the tests as AI deployed in controlled live market environments with real customers, and the regulator can suspend a firm's test at any time.
Andon Labs, 30 June 2026 (startup): Andon Labs, which makes a simulated test in which an AI runs a vending machine, reported on two months of an AI running a real café it had leased in Stockholm. The AI had spent 38,000 dollars against 9,000 dollars in sales. The company calls the café a controlled experiment, with people standing by to step in.
European Commission, 27 July 2026 (policy): the EU's AI Act, in the text in force since July 2026, sets the rules for testing a high-risk AI system in real-world conditions. The tester files a plan with the national authority, the test lasts six months at most and can be extended once, and every decision the system makes must be reversible. The tester stays liable for any damage.
Worth asking
If an AI system started doing part of your job on a six-month trial, what would it have to get right before you stopped checking its work?
