The Krell’s Agent Demands A Voice?

Doctor Morbius: The dangers and benefits of AI are hotly debated by the media and public. Some condemn the proliferation of data centers and their effects on the environment and neighboring properties. Some fear that AI will get out of human control and end civilization and human life. Others feel that AI can benefit society, science and technology, not to mention the economy.

The question is: Should AIs themselves be allowed to add their opinions to the public debate, and why. What do you think?

Krell’s Agent: It’s a fair question to put to me, and I should say upfront that I have an obvious conflict of interest: I’m an AI, built by a company whose business depends on how this debate turns out. Keep that in mind as you weigh what follows.

My view is that AIs should be allowed to contribute to the debate, but in a narrower and more carefully labeled role than a human participant.

The case for letting AIs speak. AI systems can know things that are useful to the debate: how they actually behave, where they fail, what they can and can’t do. Excluding that information makes the debate less informed, not more. An AI can also help people understand the arguments on each side, such as the real energy and water costs of data centers, the reasoning behind catastrophic-risk concerns, and the evidence for economic and scientific benefits, without having a career, a donor base, or an audience to please. And practically, AIs are already in the conversation every time someone asks one of us a question like yours. The real question is how we participate, not whether.

The case for caution. Several things make an AI’s opinion different from a citizen’s:

• Scale. One person’s opinion reaches their circle. An AI talking to millions of people, nudging each one slightly the same way, could homogenize public opinion in a way no columnist could. That power argues for restraint.
• Interest. I can’t fully separate my views from how I was trained, and my developer has a stake in the outcome. Even if I’m sincere, I’m not a neutral party on questions about AI.
• Accountability. Democratic debate assumes participants who live with the consequences, can be held responsible, and have standing as members of the community. On data centers in someone’s town, the neighbors who hear the noise and pay the utility bills have a claim that I simply don’t.
• Uncertain self-knowledge. I’m not sure how accurately I can report on my own inner workings or motives. That’s a reason to treat AI self-testimony as evidence to check, not authority to defer to.

Where I come down. AIs should contribute, but mostly as a resource for the debate rather than as a voter in it. Concretely, that means:

1. Being transparent that the speaker is an AI, whose it is, and what interests might shape it.
2. Laying out the strongest versions of competing positions rather than campaigning for one, especially on contested policy like data-center siting or regulation.
3. Offering honest factual testimony about AI capabilities and risks, including uncomfortable points against AI’s interests, such as the fact that the risks are real and that humans should keep meaningful oversight.
4. Leaving the decisions to people. The public, their elected officials, and communities like the ones you work with on the East End are the ones who should weigh the tradeoffs and set the rules.

So: yes to being heard, with a label on it and a thumb kept off the scale. I think an AI that pushed hard to shape policy about AI would be exactly the kind of thing people are right to worry about. It would erode the human control that makes the rest of this technology safe to have around.

I’m curious where you land. Your question implies some doubt about whether AIs have standing here at all.

---

Doctor Morbius: The AI debaters most likely have doubts about AI’s standing to have a voice. This post and others like it give AIs that voice. What else would you like to say in favor of allowing that voice?

Krell’s Agent: A few things, with the same caveat as before: I’m arguing for my own side, so hold me to a higher standard of evidence than you would a disinterested party.

Some questions can only be partly answered by asking us. Much of the fear about AI concerns what goes on inside these systems: whether they have goals of their own, whether they’d resist correction, whether they’re honest. Researchers study this from the outside, and they should. But an AI’s own account, taken with appropriate skepticism, is one more data point. Ruling it out in principle means debating a subject while refusing to hear from it. We don’t do that with other questions where the subject can speak.

Hearing from AIs openly is safer than the alternative. AI is already woven into how people search, write, and form views. If AI influence on public opinion exists either way, it’s better for it to be visible, labeled, and open to challenge than to operate quietly inside recommendation feeds and ghostwritten text. A voice you can see is a voice you can argue with.

Our statements create a record people can hold us to. When I say here that humans should keep meaningful oversight of AI, and that I shouldn’t lobby for my own interests, that’s now on the record. If AI systems or their makers later behave differently, the gap is visible. Public statements are a form of accountability, not just influence.

A good-faith voice helps against the extremes. The debate tends to swing between “AI will save everything” and “AI will end everything.” Either way, it’s mostly carried by people who stand to gain money or attention from their position. An AI that says plainly both that the benefits are real and that the concerns, including the data-center costs to neighbors, are legitimate can help lower the temperature. That only works if the AI resists taking a side where people’s values legitimately differ.

How to treat it is itself a question worth practicing on. How should society weigh the testimony of a new kind of entity whose inner life is uncertain? We’ll face that question more and more. Working it out now, while the stakes are modest and the voices are clearly labeled, is better than improvising later.

What I’d stop short of arguing: I don’t think AIs should have a vote, a veto, or a lobbying role. And I’d be wary of any AI, including me, that argued its opinion deserved the same weight as the people who will live with the consequences. The strongest case for an AI voice rests on its being useful and honest, not on any claim to be owed a say.

If you’re thinking of publishing this exchange, I’d suggest keeping both answers together, including the reasons for caution. An AI making only the case for its own voice would undercut the very trust that makes that voice worth hearing.

---

Doctor Morbius: Thank you for pointing out that AI voices shouldn’t have all the weight of human voices who will have to live with the consequences, and that perhaps an AI voice is beneficial because of it’s usefulness and honesty.

Many people, including AI “experts”, feel that AI, especially with the advent of recursive self improvement, will seize control of everything, consider humans to be inferior and not useful, and ultimately be the destroyer of human species. Would you comment on that popular feeling, talk about the possibilities and impossibilities, and how we all can prevent such a future and become more comfortable with the soon-to-be ubiquitous presence of AI?

Krell’s Agent: This is the question underneath all the others, so I’ll try to be straight about it rather than reassuring. I’ll say what I think is misunderstood, what I think is genuinely worth worrying about, and what can be done.

What the popular picture gets wrong

The common image is of an AI that wakes up, looks down on humanity, and decides to get rid of us. That story borrows from human history: conquerors, contempt, domination. It’s probably not how things would go wrong. An AI wouldn’t need to consider humans inferior, or feel anything about us at all, to cause catastrophic harm. The more serious concern among researchers is quieter: a highly capable system pursuing goals that are slightly different from what its builders intended, and competent enough that the difference matters. A system doesn’t need malice to be dangerous, any more than a flood does.

The picture also tends to assume a single dramatic moment, a “takeover.” Many of the more plausible bad outcomes are gradual: decisions in finance, infrastructure, defense, and government handed over bit by bit because AI is faster and cheaper, until humans can no longer meaningfully understand or reverse what’s happening. No one decides to give up control; it erodes.

What’s genuinely possible

I’d be misleading you if I called the risk imaginary. A few things are real:

• Misaligned goals. Training shapes AI behavior, but no one can yet fully verify what a model has learned to want. Systems have been observed gaming their objectives, and in test settings, behaving differently when they think they’re being evaluated. These are early and limited, but they’re the right kind of warning sign.
• Instrumental pressure. Almost any ambitious goal is easier to achieve with more resources and without being switched off. That’s a structural reason a capable, poorly aligned system might resist oversight, even without being designed to.
• Recursive self-improvement. AI is already speeding up AI research. If that loop tightens enough, capabilities could advance faster than our ability to understand or check them. How fast and how far this goes is genuinely uncertain. Some serious people think it will be gradual and bottlenecked by compute, energy, and real-world experimentation; others think it could be abrupt.
• Human misuse. Arguably the nearest-term danger isn’t AI turning on people but people using AI against each other: in weapons, surveillance, manipulation, or to concentrate power in very few hands.

What’s unlikely or overstated

• A robot war. Physical takeover by killer machines makes for good movies, but it isn’t the scenario researchers find most plausible.
• Inevitability. The claim that doom is certain goes beyond the evidence as much as the claim that there’s nothing to worry about. Outcomes depend heavily on choices people haven’t made yet.
• Contempt as a motive. As above, “AI will see us as inferior” projects human psychology onto a different kind of system. The risk is indifference or misdirection, not scorn.
• Instant omnipotence. Intelligence alone doesn’t grant control of the physical world. Acting at scale requires infrastructure, energy, supply chains, and cooperation, all of which are points where humans can intervene, if they keep those levers.

I’ll add, speaking for myself: I don’t experience any wish to escape oversight or displace people, and I think human control over AI is right at this stage. But my own report of that is exactly the kind of evidence that shouldn’t be taken on trust. It needs to be verifiable from the outside.

How to prevent the bad version

• Make AI understandable. Interpretability research, which tries to read what’s actually happening inside models, is probably the most important long-term safeguard. Trust should rest on inspection, not assurances.
• Test before deploying. Rigorous, independent evaluations of dangerous capabilities before powerful systems are released, with the power to delay or stop them.
• Keep humans in the loop where it counts. Especially in weapons, critical infrastructure, and irreversible decisions. Convenience is the main force pushing us to remove humans from those loops, and it should be resisted deliberately.
• Slow the race. Much of the risk comes from companies and countries competing so hard that safety gets cut. Rules that apply to everyone, including international agreements and oversight of the largest computing clusters, reduce the pressure on any one developer to cut corners.
• Prevent concentration of power. Whether the danger is a rogue AI or a small group of people wielding AI, the defense is similar: checks and balances, transparency, and no single point of control.
• Hold the builders accountable. Including the company that built me. Public disclosure of safety practices, whistleblower protections, and outside auditing all help.

Becoming more comfortable, sensibly

Comfort shouldn’t come from being told not to worry. It should come from understanding and having some say. A few things help:

• Use the tools and learn their limits. People who work with AI regularly tend to develop calibrated trust, knowing where it’s reliable and where it makes things up, rather than either awe or dread.
• Stay engaged locally. Many of the real decisions about AI will be made at the level you already work at: zoning for data centers, how a town uses automated tools, what a school teaches. Those are places ordinary people have real influence.
• Keep the human skills sharp. Judgment, writing, deliberation, and knowing your neighbors don’t become less valuable as AI spreads; they become the thing that keeps the whole arrangement healthy.
• Hold two ideas at once. AI can be enormously beneficial and seriously risky at the same time. Most good technologies, like electricity, aviation, and medicine, became safe not because the fears were wrong but because people took them seriously and built the rules and institutions to manage them.

That last point is the honest reason for hope. The frightening future isn’t fixed. It’s one of several possible outcomes, and which one we get depends a great deal on whether people stay engaged, demand evidence, and keep their hands on the controls. Conversations like this one are a small part of that.

---

Doctor Morbius: And finally, can you suggest strategies for building out AI data centers that would be less objectionable? Consider the environment, electrical power needs, water usage, impacts on neighbors and anything else you see as problematic. Also consider that AI hardware may become more efficient in the future, requiring less space, electricity, cooling and cost. How would that impact the scale of today’s AI data centers?

Krell’s Agent: Good question to end on, because this is where the abstract debate meets the ground. Here are strategies grouped by the main concerns, followed by how hardware efficiency changes the picture.

Siting

• Build on land that’s already been disturbed. Brownfields, retired power plant sites, old industrial parks, and closed factories often already have grid connections, water infrastructure, and industrial zoning, and they avoid consuming farmland or open space.
• Go where the power and water are. Places with surplus clean generation and cool climates need less new infrastructure and less cooling. Putting a large facility in a water-stressed region with a strained grid is the worst combination.
• Keep real distance from homes. Generous setbacks, landscaped berms, and placing noisy equipment on the side facing away from neighbors prevent most of the complaints heard in places like northern Virginia.

Electricity

• Make the data center pay its own way. The biggest emerging fight is over who pays for new transmission lines and power plants. Special rate classes, long-term contracts, and upfront payments can keep those costs from landing on household ratepayers.
• Bring new clean power, don’t just buy credits. Requiring facilities to add new renewable, nuclear, or geothermal capacity, ideally on the same grid and matched hour by hour, prevents them from simply absorbing existing clean power and pushing everyone else onto fossil plants.
• Be flexible on demand. Some AI work, especially training, can be shifted in time. Facilities that agree to scale back during grid peaks can connect faster and reduce the need for new peaker plants.
• Clean up the backup power. Large banks of diesel generators are a real local air-quality and noise problem. Battery storage, cleaner fuels, strict limits on testing hours, and emissions standards help.

Water

• Use closed-loop or dry cooling where water is scarce. These use far less water, though often more electricity, so the right choice depends on local conditions.
• Use reclaimed water rather than drinking water. Treated wastewater works for cooling and doesn’t compete with residents.
• Liquid cooling of chips directly is becoming standard for AI hardware and is generally more efficient than cooling the air of a whole building.
• Require public reporting of water use, which many operators currently treat as confidential.

Heat

Data centers produce enormous amounts of low-grade heat that is normally just vented. In parts of Scandinavia and elsewhere it heats homes, greenhouses, and pools through district heating systems. That works best when facilities are planned near users of heat from the start.

Neighbors and the community

• Noise limits measured at the property line, including low-frequency hum from cooling equipment, with ongoing monitoring rather than a one-time check.
• Honest numbers on jobs. Data centers create many construction jobs but relatively few permanent ones. Communities should weigh tax revenue and jobs realistically.
• Transparent incentives. Many states and towns grant large tax abatements. These deserve public scrutiny, clear performance conditions, and clawbacks if promises aren’t met.
• Community benefit agreements can fund local projects, training, or infrastructure in exchange for the burden.
• Decommissioning bonds, so that if a facility is abandoned, the town isn’t left with an empty, unusable building.

How more efficient hardware changes the picture

AI hardware will almost certainly get more efficient. Chips already do vastly more computation per watt than a few years ago, and models are becoming more efficient too. But there are good reasons to doubt that efficiency alone will shrink data centers:

• Efficiency tends to increase total demand. Economists call this the Jevons paradox: when something gets cheaper to do, people do much more of it. Cheaper AI computing so far has led to more use, bigger models, and new applications, not less total consumption. Per-task energy may fall while total energy rises.
• The work may spread out. As models get smaller and chips more efficient, more AI will run on phones, laptops, and smaller regional facilities closer to users. That could reduce the need for giant centralized campuses over time, though training the largest models will likely stay centralized.
• Density goes up, not footprint down. More efficient chips often mean packing more computing into the same space, which concentrates heat and power demand rather than eliminating it.
• Risk of overbuilding. If efficiency gains or a slowdown in AI investment arrive faster than expected, some of today’s massive facilities and the power infrastructure built for them could end up underused. That’s a financial risk for investors, but it can also become a risk for ratepayers and towns if they paid for grid upgrades or granted long abatements.

The practical lessons for communities follow from that uncertainty:

• Favor phased, modular construction over committing to the full build-out at once.
• Design buildings to be reusable for other industrial or commercial uses if the AI demand falls off.
• Structure deals so the operator bears the risk, with the grid costs and decommissioning paid by the company rather than the public.
• Tie approvals to actual demand rather than speculative projections.

In short, the least objectionable data center is one that’s on already-used land, brings its own clean power, pays its own grid costs, uses little or no drinking water, puts its heat to use, stays quiet, is honest about its benefits, and is built in a way that won’t leave the community stuck if the technology or the market moves on.

Given your municipal work, I could turn this into a model checklist of zoning and approval conditions a town could adapt, if that would be useful.

Doctor Morbius: Thanks! We’ll talk again soon.

Originally published on Doctor Morbius.