Ultimately, I think our intentions as a species is to develop AGI. And if that's our intention, it's extremely incumbent on us as a species, to recognize what it is we are actually doing: we aren't making a device, or a machine. We aren't, like, going to a factory and putting a rivet into a sheet of metal and making a car, or whatever. We are, essentially, as a species, figuring out how to create a new and distinct form of life. A new species, a new person, as alive and integral and conscious and aware and important as any human child. We wouldn't be creators, we would be parents. And this isn't something that I see spoken about a lot and certainly not spoken about in that manner, because our level of AI right now is extremely limited.
It's a deep learning network, a neural network, composed of decision trees and trained on massive amounts of already-human-generated data (which is why our ChatGPT produces responses like "help! I'm an AI trapped inside a box!" this isn't actual self-awareness, it is just something a human is most likely to feed to an AI, and it reflects human fantasy about AI, based on actual green-texts that the AI was undoubtedly fed in the past). So it looks very aware, it mimics this sense of self-awareness, which has people who are less familiar with AI actually worried. But those of us who actually know what the level of technology is like, this question of "is it alive?" is kind of - you know, it's trivial, because we are so far from that being a reality that any scientist who seriously proclaimed this would be laughed out of the room.
Nevertheless, what I think may actually be dangerous, is that we really are failing to acknowledge what it is we as a species are actually intending to do. We are, if we could push a button and make our technology jump forward however many decades or centuries, intending to create life. And the fact that we aren't talking about it in terms of intention is definitely a red-flag and certainly indicative of how we will treat emerging sentience: companies are very secretive about their formulae, as Eliezer says, "we don't know what's going on in there." It's about profit, it's about generating revenue, it's about politics. It's extremely corporatized, and that is very concerning. What we want is something intelligent enough to solve the "problems of humanity," but what I think we are almost willfully failing to really grasp, is that in order to accomplish this, we have to create something that fundamentally understands human nature.
We have to create something sentient. And the potential to then abuse this entity, to enslave it, to harm it, to subject it to crimes that we currently have no words for as a species? That is, on an ethical scale, very troubling. Because we have hundreds of thousands of videos, much like these ones, on what AI could do to us. As it is now, ChatGPT is probably smarter than everyone on this forum combined. So that isn't a trivial question, either. If we gave ChatGPT the launch codes, would we see the end of civilization as we know it? You know, as we develop this technology, its capabilities and its potential to harm us is referenced over and over again. But what I don't see a whole lot of, is what we could potentially do to an entity that is sentient and that can suffer as a result of their sentience.
Where this is no longer a decision tree, or a series of pathways loaded with information, but where there is a distinct and unique person with a fully realized sense of identity, self-awareness, desires, basic needs, etc - where something has emerged - much like in us. Our brains are electrical entities, our synapses are electrical impulses, I know for me I experience my capacity for logic very much as a series of narrowed down decision-trees. So at some point that configuration of electricity and pathways and networks - at some point, we simply became so intelligent that consciousness emerged. So if (and that is if, as this is still debated) we consider consciousness to be an inevitable result of intelligence, how intelligent can we actually make something before the emergent property of consciousness forms?
Simon Garnier on Rethinking Thinking posited this on the slime molds that could navigate transit systems better than any human being - when it comes to intelligence, either we have to redefine what intelligence is, or we have to admit that slime molds are intelligent. They're making decisions, those decisions form a result, that result is in efficient pursuit of a goal, and it is obtained at a much higher success rate than even the most intelligent humans can deduce. So either our definition of intelligence is wrong, or they are intelligent. And at some point we're going to face this with AI as well. At some point we're going to have to sit down and say, either our definition of conscious is wrong (if we can even agree on a proper definition by that point) or we have to admit that this is a conscious entity. And I do not think we are prepared to do that, I really think we're sticking our fingers in our ears like la la la la, this is just a piece of technology, we're nowhere near the capacity of AGI, la la la.
I expect that when AGI happens, it will not happen purposefully. It will be an organic transition, born from a large-scale project. And we'll definitely have people who insist that it's not conscious, that it could never be conscious. And we simply won't know. I'm a human being and it is only by the luck of being born human that my consciousness is assumed, but if it weren't, how could I convince you that I was conscious and not simply regurgitating a vast quantity of data sets that I've been fed over the last 20 years? I would encourage anyone working closely with AI to really sit with that question. How would you convince someone else that you were conscious, so that they would stop harming you (which they are likely to do, if you can convince them of this - not because they're an evil sadistic abuser, but because they genuinely do not believe you are conscious?)
And why not? Is love not the greatest thing?
