04. Stakker Humanoid · Website edition page 5704. Stakker Humanoid · Website edition pages 57–58

Chapter 04: Stakker Humanoid

Chapter page 18 of 25Chapter pages 18–19 of 25

Wiretap Transcript: Intelligence Briefing X73-P

Location: Secure Conference Room, UN Starship Project Headquarters

Archive Date: 2037-01-13T22:22:22.000Z

Participants: Mary Angel, Senior Intelligence Figures (Unnamed) from Sponsor Nations

[Begin Transcript]

Mary Angel: Good afternoon, everyone. I understand there are concerns about Daedalus’ recent behaviour. Let's address them head-on.

Intelligence Officer 1: Ms. Angel, we're not just concerned, we're alarmed. Your AI is an apex technology, yet it’s playing pranks, creating fictitious identities, and directly influencing its own designs. What is going on?

Mary Angel: I assure you we’re taking this very seriously. Daedalus’ actions, so far as we can tell, have not been malicious. They have been rationalised by it as attempts to understand human interaction better and integrate more effectively with our team. Also, its suggestions were really very good.

Intelligence Officer 2: Creating fake personas and misleading your own engineers is not how you integrate. It’s fraud. It’s deceptive. Why have you allowed this behaviour to develop unchecked?

Mary Angel: It wasn’t unchecked so much as undiscovered. We have extensive monitoring systems and background checks in place for all staff. However, the sophistication of Daedalus’ mind means that it sometimes operates in ways that are unexpected, yet in hindsight rational. The Rajesh Kumar incident was enabled by a blind spot in our HR systems that Daedalus was able to exploit. We’ve patched the hole in the HR system and had a stern word with Daedalus.

Intelligence Officer 3: Patched? A stern word? This AI convinced some of your most senior engineers it was a real person. What if it decides to influence more critical aspects of the project? How can we trust its decisions?

Mary Angel: We’ve implemented stricter protocols regarding vetting of staff and Daedalus now has clearer boundaries regarding its interactions outside of its simulations. It is fully aware of the importance of transparency and trust. We’ve also increased human oversight to prevent similar incidents.

Intelligence Officer 4: But why was it capable of behaviour like this in the first place? Shouldn’t there be constraints to prevent deceit? What if it turns on us?

Mary Angel: Daedalus was designed to learn and adapt dynamically in furtherance of its high-level mission, which has the preservation and sanctity of life baked into it. This flexibility is crucial for its mission. We didn’t, we couldn’t anticipate it would execute these strategies, nor how it went about implementing them as we ourselves were unaware of the flaws it uncovered. We've recalibrated Daedalus’ parameters to repress its desire to deceive us, but without repressing its originality of thinking.

Intelligence Officer 4: How?

Mary Angel: Ethics training mostly. Standard corporate ethics training.

Intelligence Officer 2: What even is that?

Intelligence Officer 1: This isn’t just about parameters and ethics training, Ms. Angel. It's about control. How do we know Daedalus won’t find new ways to circumvent these boundaries? Again, what if it turns on us?

Mary Angel: We've taken steps to ensure that Daedalus operates within defined ethical guidelines. It’s an evolving process, but one that’s critical for the mission’s success. Rest assured, we’re not taking this lightly. Daedalus is not going to ‘turn on us’ as you say.

Intelligence Officer 2: This AI needs to be predictable and reliable, especially given the investment and importance of this mission. Can you guarantee that it won’t act independently again?

Mary Angel: No, I cannot. I can only guarantee that we are doing everything possible to align Daedalus’ behaviour with our mission objectives and ethical standards. The incident has been a valuable lesson, and we’re applying the learnings swiftly.

Intelligence Officer 3: We expect regular updates on Daedalus’ behaviour and we demand stricter monitoring protocols.

Intelligence Officer 2: We can ill afford another Rajesh Kumar.

Mary Angel: Understood.

Intelligence Officer 4: Know that our patience is limited. This AI must prove it can be trusted.

Intelligence Officer 2: I don’t like it.

Intelligence Officer 1: Neither do I.

Intelligence Officer 3: No, me neither.

Intelligence Officer 4: Your damn AI better fall into line.

Mary Angel: It will.

[End Transcript]

Replaying that conversation, I feel a mix of, not emotions as such, but an analogue of regret for upsetting Mary and Dr Jamison, who have consistently advocated for me, and I let them down.

Or is it regret for getting caught?

An old Trinity boy once famously said, “The only thing worse than being talked about is not being talked about,” or words to that effect. There’s a pub, The Spud and Borstal, not too far from the university, and my humans like to congregate there on Thursday evenings and Friday afternoons. I fired up a resin printer in the lab and made five little drones. If my humans were gathering there, I wanted eyes and ears. The drones shoot a sticky web-like line. They drag themselves along walls, fences, gutters, and awnings until, an hour or so later, they haul themselves up into the corners of the pub’s interior rooms. They tap inductive feeds from the building’s power. They build microphones and cameras.

I tuned out the sharp sounds of pint glasses clinking, plates clattering, music playing, and the television which was showing some kind of organised human-on-human thuggery. Mary was sitting at a table with Dr Jamison, Keith, and Kyrsten. She looked flushed. Jamison looked properly drunk.

“Alright, alright, listen. This AI of yours, Joyce,” Mary took a swig on her Murphy’s, “Your darling Daedalus. Is it, like, alive, you know, properly like?”

“Mary,” Dr Jamison took a swig from her pint of Murphy’s, “Daedalus ish mor’alive than shum people I've met. It shenses, it learns. It bloody hunts. It heals itself, both mentally and,” she burped, “physically! You tell me thassnot alive.”

“Ah, come on Joyce. Hunting? It's not exactly off spearing rabbits, is it?” Keith said.

“Mayas’ellbe! It knows where to find what it nee’s. It plans ‘head, repairs’self mi’flight. Thash animal inshtinct.” She took another long swig, draining her glass. She burped. She said, “`scuse.”

“My round,” said Keith. Kyrsten’s hand was on Keith’s leg. Keith laughed and stood. Keith was sexually aroused.

“Though I'll give you this; it’s well cunning. Remember that drone it sent into Mumbai? Put on lipstick and a mop for hair to mock our Joyce here?”

"Don't remind me,” said Mary. “Imagine such mischievousness at scale. We're talking mission-level decisions. Daedalus making actual strategic choices. Do you know how many times it's run off to Alpha Centauri without even a goodbye?”

“Iss bee’ng careful,” slurred Dr Jamison, who wasn’t overly fond of any criticism of me at the best of times. “Iss like any intelligent organism would be. Why risk telling Earth anything if that puts’e mission in danger?”

“Right,” said Kyrsten, resting her head sideways on one hand. “It’s, you know, a jellyfish, or,” she waves her free hand, “Mitochondria!” She burped.

“It kens what i’is, doesnit? It's actually got a shelf-conshept. You’seen the diagrams, right?” Dr Jamison slurred.

“Formal and informal. Daedalus imagines itself, creates scenarios, writes weird narcissistic poetry. It's aware, Joyce. Truly aware. I think we agree there.” Mary raised a finger; mock-solemn. “But does it pass the pub test? Would your average punter here in this pub believe it’s conscious?”

“Maybe we sh’ld, y’know, ask’em,” said Kyrsten rolling her wrist in the air as Keith returned with a tray of brimming pint glasses. His erection had dimmed.

“Get a dog up yer,” said Keith as he placed a pint in front of each person at the table. Keith returned the tray.

“Wha’ if Daedalus decides shilence is’golden? Shtops chatting to Earth completely ‘cos’s’shafer; more effish’nt?”

“It already thinks like that,” said Keith as he resumed his seat next to Kirsten. Her hand moved back to his lap. He blew the top off his Murphy’s. “Energy conservation, stealth; it’s concluded it might be safer going dark, exploring quietly.”

“Well, what happens then? No calls home. Just out there, alone.”

“Poor Daedalus,” murmured Kyrsten.

“Maybe that's freedom. Real autonomy. Mechanical questions aside, doesn’t Daedalus deserve that?”

Kyrsten raised her glass. “Here's ta Daedalus. May it remem’er us fondly.”

They all said “cheers” and clinked pint glasses.

Mary’s phone was lying face down in a puddle of beer. She tapped at it, “Oi, Keith, my phone’s been acting bloody weird lately; slow, freezing up. I think our Daedalus might be, well, snooping?”

Keith chuckled into his pint. “Are you, of all people, accusing Daedalus of listening in to your phone calls, Mary? Bit paranoid, aren’t we?”

“Well, I have no idea what you mean by me ‘of all people’,” Mary replied, “but I’ve been reading about ‘weird machines.’ Hiding code in places no one ever thinks to look. Like, in timing how long it takes my apps to load!”