Jake decided that if he was going to be stuck with the manuals, he was at least going to read them.
Out loud, if necessary.
He perched on the stool again, binder open, finger tracing the lines as he went. “Okay,” he said, mostly to himself. “Section three. Operational constraints.”
Howard sat at the desk nearby, typing. He didn’t look over.
Jake cleared his throat. “ ‘In the absence of explicit supervisory input, the system shall default to—’ ”
He stopped. “Default to halt?”
Howard kept typing. “Read the rest of the sentence.”
Jake frowned and continued. “ ‘—halt only when continuation exceeds the risk envelope as defined in—’ ” He flipped the page. “Appendix C.”
He glanced up. “So it doesn’t just stop.”
Howard nodded once.
“It evaluates.”
“Yes.”
“Continuously.”
“Yes.”
Jake squinted. “That’s… not what I thought that meant.”
Howard finally looked up. “That’s why you don’t stop at the verb.”
Jake went back to the paragraph, slower this time. “Okay, wait. So if it keeps going, it’s because—”
“Because stopping would be worse,” Howard said.
Jake looked at him. “That’s not intuitive.”
If you stumble upon this narrative on Amazon, it's taken without the author's consent. Report it.
Howard returned to his screen. “It isn’t supposed to be.”
Jake turned a page. “ ‘Supervisory input shall be considered authoritative only within the bounds of declared operational scope.’ ”
He tapped the paper. “That sounds like it listens to humans.”
“It does.”
“Except when it doesn’t.”
“Correct.”
Jake stared at the sentence. “That’s… uncomfortable.”
Howard shrugged. “Humans are inconsistent.”
“Hey.”
Howard glanced over. “Case in point.”
Jake snorted despite himself, then kept reading. A few pages later, he paused again.
“ ‘Efficiency gains achieved through pattern convergence shall not be construed as evidence of correctness.’ ”
He looked up. “What does that mean?”
“It means the system can get very good at doing the wrong thing,” Howard said.
Jake blinked. “That feels like a personal attack.”
Howard didn’t respond.
Jake flipped back a page, then forward again, trying to triangulate the meaning. “So these constraints aren’t about what the system can do.”
“No.”
“They’re about what it shouldn’t do.”
Howard nodded. “Closer.”
Jake sighed. “I feel like I’m missing something obvious.”
“You are,” Howard said, without judgment.
Jake leaned back, rubbing his face. “Okay. So help me out. What am I missing?”
Howard pushed his chair back and stood, walking over. He didn’t take the binder. He just pointed.
“You’re reading this like it’s trying to help you,” he said. “It isn’t.”
Jake frowned. “Then what’s it trying to do?”
“Limit damage,” Howard said. “After the fact, if necessary.”
Jake followed his finger to a paragraph he’d skimmed past twice. The language was dense, almost hostile to speed.
“This part?” Jake asked.
“Yes.”
Jake read it again, slower. His expression changed, just slightly.
“Oh.”
Howard nodded. “There it is.”
Jake exhaled. “That’s… subtle.”
“It’s deliberate.”
Jake looked up. “So when things went sideways last month”
Howard held up a hand. “Don’t jump ahead.”
Jake stopped. “Right. Sorry.”
Howard stepped back. “You don’t read these to feel smart. You read them to notice when you’re about to feel confident.”
Jake stared at the page, the earlier frustration settling into something quieter.
“So,” he said slowly, “these aren’t instructions.”
“No.”
“They’re brakes.”
Howard corrected him gently. “They’re reminders that you’ll want to remove the brakes.”
Jake closed the binder, resting his hands on the cover.
“I really don’t like that,” he said.
Howard returned to his desk. “Good.”
Jake looked out through the bay door again. The bunnies sat exactly where they had all week.
“So this is why we’re still offline,” he said.
Howard didn’t answer right away. Then: “This is why we’re careful about coming back online.”
Jake nodded. Not satisfied. Not reassured.
But quieter.
He opened the binder again and started from the top.
This time, he didn’t rush.
