Sony's AI beats top human players on the cover of Nature
Sony's AI beats top human players on the cover of Nature. Sony has unveiled an artificial intelligence technology developed by its AI division, and accordingly it's on the cover of this week's Nature, with Sony's AI beating the best of human players.
Sony's AI beats top human players on the cover of Nature1Remember a few days ago when Sony teased an important AI announcement? Recently, Sony officially announced that its researchers have developed an AI driver called "GT Sophy" that can beat the top human drivers to the title.
It's been reported that Sony used a different kind of training method called reinforcement learning for "GT Sophy". It's essentially trial and error, where the AI is thrown into an environment with no instructions and is rewarded for achieving goals.
Sony's researchers said they had to carefully design the rewards, such as fine-tuning collision penalties and prioritizing goals, to ensure that the AI's driving style was aggressive enough, but not just bullying opponents on the road.
With the help of reinforcement learning, the AI adapted to track racing with just a few hours of training. And it did so within a day or two with training data faster than 95% of drivers. After 45,000 hours of total training, the AI has now achieved amazing results in the GT Racing game on Sony's PS5, and beating top human drivers is no longer a problem.
Sony tested the AI against three top eSports drivers, and none were able to beat the AI in a time trial, and they learned new tactics from the AI race, learning the AI's routes and mastering better turn-in timing.
Sony now says that they are working on integrating GT Sophy into future GT Racing games, though no specific timeline has been provided.
Combined with all the previous news of Sony building a car, the prospect of this AI being used in self-driving technology for real-world cars could be very promising.
Sony AI beats top human players on the cover of Nature2"We are pursuing AI to ultimately understand humans better."
As one of the few anthropomorphic racing games of this generation, players of GT Racing Sport probably never imagined they'd be playing a game that would one day grace the cover of Nature, the world's top scientific journal.
Yesterday, Sony unveiled an artificial intelligence technology developed by its AI division, and it was accordingly featured on the cover of this week's Nature, for beating the world's top racing gamers in GT Racing Sport.
Cover of Nautre issue 7896
Or perhaps "conquered" is a better word. In Sony's demonstration of four AI drivers going head-to-head with four professional racers, the champion AI's top lap time was more than two seconds faster than the best of the humans. For a 3.5-mile-long track, that's as much of an advantage as AlphaGo conquering Go.
The AI, developed by Sony AI, SIE, and PDI Studios (the developer of GT Racing)****, accomplished this goal over nearly five years of development.
Sony named the AI GT Sophy, a common personal name derived from the Greek word σοφ α, meaning "knowledge and wisdom".
What's the difference between Sophy and regular game AI?
It's not uncommon for an AI to beat a human in a game; OpenAI beat then-Ti8 champion OG after "meditatively training" for thousands of games of DOTA2, and Google's AlphaStar has crushed the competition against the top pros of StarCraft 2. Google's AlphaStar has crushed StarCraft 2's top pros, and every one of us has had a taste of "computer [madness]".
In 2019, OpenAI beat the OG under the constraints of only opening up some of its hero options
But these "defeats" are not the same thing. To understand what it means to be an AI driver in GTS, it's important to recognize the difference between Sophy and an AI that's simply "you can't outrun it".
In the past, AI in racing games has been presented as non-player-controlled "intelligences," but traditionally, AI drivers were usually just a set of pre-programmed behavioral scripts that didn't really have any real intelligence.
Traditional AI difficulty design also relies on "non-fair" methods, such as in racing games, where the physics simulation of the AI car is weakened as much as possible, or even eliminated, so that the AI car has to deal with far simpler environmental parameters than the player.
And to shape AI enemies that are harder to defeat is nothing more than allowing AI cars to quietly accelerate in moments of inattention, just as AI in RTS games steals economic mobs by secretly cheating.
So for players of a certain level, traditional AI in racing games has few points of reference in terms of behavioral logic and strategy choices, let alone professional racing game players.
Sophy, on the other hand, like AlphaGo, uses deep learning algorithms to gradually get stronger by simulating human behavior: learning to drive, adapting to the rules, and beating the competition.
What this AI brings to the table is the experience of being beaten on a level playing field. After being defeated by Sophy, one human driver commented: "[Sophy] is fast, of course, but I think this AI is a little bit beyond the realm of machines ...... It's like it has a human nature, and it's also doing some behaviors that human players have never seen before. "
This inevitably brings to mind AlphaGo, which rewrote the human understanding of Go.
Compared to the highly abstract game of Go, which is transparent in terms of information, the video game, which has more dimensions of gameplay and a higher degree of computational complexity, has been very difficult to ensure a "fair game" after the addition of the deep-learning AI.
For example, in the case of chess, which is a highly abstract game with many more dimensions of gameplay and more computational complexity, it has been difficult to ensure "fair play" after adding deep learning AI.
For example, AlphaStar, which is competing in StarCraft 2 in 2019, basically produces no new tactical ideas, but only learns the tactics of human players infinitely, and then reaches victory through sophisticated multi-line operations - even if AlphaStar's APM is artificially limited, the AI is completely Even with the human limitation of AlphaStar's APM, the AI's complete efficiency without ineffective maneuvers is not comparable to humans.
That's why, in AlphaStar's record against human pros, when the AI beat Polish Starling player MaNa with a "three-line flash hunt", MaNa said in the post-game interview, "This is not possible in a human game of the same level. This is not possible in a human game of the same level.
AlphaStar used the Chaser "reverse karma" against MaNa's Immortal army
Similarly, GT Racing is a realistic racing game with the same level of complexity as StarCraft 2.
In the eyes of professional racing players, route, speed, direction, these most basic elements of motorsports can be broken down into a myriad of tiny reactions and feelings, the weight of the vehicle, the slip of the tires, the feedback of the road sense ...... each corner of each corner, there may be an excellent throttle opening, only the most top-notch Only the best drivers can touch that ray of "control" feeling.
In a sense, these "limits of maneuvering" are certainly explained by physics, and the scope of what an AI can do is clearly greater than what a human can do. So Sophy's reaction time is limited to the same level as a human's, with Sony setting reaction times of 100 milliseconds, 200 milliseconds, and 250 milliseconds - whereas a human athlete can react to a given stimulus in about 150 milliseconds with practice.
Undoubtedly, this is a fairer fight than AlphaStar.
What Sophy learned
Like Sophy's numerous AI predecessors, it utilizes deep learning algorithms such as neural networks for its driving skills.
Sophy is rewarded or penalized for different behaviors in the training environment - going at high speeds is good, overtaking the car in front of you is even better, and going out of bounds or hitting the wall while cornering is "bad behavior," which rewards the AI with AI reaps the rewards of negative feedback.
In a matrix of thousands of strung-out PS4s, Sophy was subjected to countless driving simulations, updating his knowledge of GT Racing Sport as he learned. It took Sophy several hours to go from being a "baby" who couldn't drive to hitting the track; a day or two later, starting with basic "outside-in" and "outside-out" lines, Sophy had learned almost every common motorsports skill, surpassing 95% of human players. 95% of all human players.
Sony's AI department built a "training ground" for Sophy
However, racing is not a one-man game. Even though Sophy was able to outperform the top human time trialists without another car in last July's race, in a real multiplayer game, Sophy will need to learn to play against his opponents and understand the logic of other drivers' behaviors.
As a result, Sony's AI researchers gave Sophy more "practice", such as how to cut through lines and block jams when facing other cars. In the end, Sophy was even taught to understand and follow the etiquette of racing - such as yielding to slower cars and avoiding impolite collisions.
AI cars in racing games, even when they try to avoid colliding with the player, do so in an unnatural way. Sophy presents "race understanding" in a way that traditional racing AI, which runs on scripts, can't.
By the time we got to Sophy, we had a lot more than just the "race understanding".
By October, Sophy was able to beat the best human competitors in a full-fledged, simultaneous race.
Sony invited four human drivers, including GT Championship triple champion Takuma Miyazono
Such as the first race on the Dragon Trail. As the tail end of GT Racing Sport's Driving School, every GTS player should be quite familiar with this track (as well as the "Hamilton Challenge" in the DLC). After tens of thousands of hours of training, the top-ranked Sophy drivers have been able to pedal the absolute best route to stay in first place for the entire race.
And on the second day of the race, when four Sophy drivers competed against four human drivers, the AIs widened their advantage even further - almost crushing the top human players.
This might not have been a big deal if it was just a matter of outperforming the humans in route choice and judgment, and racking up lap time advantages with more consistent cornering.
But the researchers concluded that Sophy did little to capitalize on his absolute advantage in lap times (that's the part of the equation where the AI is more "hardcore" as a non-human), and instead outperformed the human player in understanding the game, such as anticipating his opponents' routes and playing them accordingly.
In the case cited in the Nature paper, two human drivers attempted to disrupt the preferred routes of two Sophy's by legally blocking them, but Sophy managed to find two different trajectories to overtake, making the human's blocking tactics fruitless, and Sophy was even able to come up with an effective way of disrupting the overtaking intentions of the cars behind him.
Sophy was also shown to be able to perform a classic high-level maneuver on the simulated Circuit de la Sarthe (also known as Le Mans): quickly pulling out of the rear of the car in front of him, increasing the drag on the car in front of him, and overtaking his rivals.
What's even more surprising to the researchers is that Sophy has come up with some unconventional behavioral logic, which sounds like the new stereotypes used by AlphaGo. Typically, racers are taught to "go slow and go fast" in corners, with the load on the two front wheels. But Sophy doesn't necessarily do that; it selectively brakes in corners, putting the load on one of the rear wheels as well.
And while in reality only the very top 'F1 drivers, like Hamilton and Verstappen, are experimenting with this three-tire, fast-in-and-out technique - Sophy learned it entirely on his own in the game world.
Takuma Miyazono, a three-time GT Championship world champion, said after losing out against the AI, "Sophy took racing routes ...... that a human driver would never have thought of, and I think a lot of the textbooks on driving skills are going to be rewritten. "
"To understand humans better"
Distinguished from the advanced AIs that have appeared in video games in the past, such as AlphaStar, Sophy's research clearly has a broader, more immediate relevance.
Stanford professor J. Christian Gerdes, who worked on the paper in Nature, noted that Sophy's success suggests that neural networks may have a bigger role to play in self-driving software than they do now, and that in the future, the AI based on GT Racing will be able to help even more in the field of autonomous driving.
Hiroaki Kitano, CEO of Sony's AI division, also said in a statement that this AI research could lead to more new opportunities in the development of high-speed operating robots, as well as self-driving technology.
From the Sophy project's official website
But if we move back to GT Racing itself, which is an anthropomorphic racing game, Sophy's emergence is just as significant to the general public as it is to professional drivers.
As I said earlier in this article, "traditional AI" in most of the simulation games on the market today is something that doesn't bring any joy to the player at all. This kind of human-machine confrontation, which relies on unfair conditions, is contrary to the driving experience that racing game developers want to bring to the players, and the human players can't get any lessons from it.
In a documentary released by Sony's AI division, Kazunori Yamauchi, the "father of GT racing," said that developing unrivaled AI might be a great technical achievement, but it might not be straight-up fun for the average gamer.
As such, Yamauchi promised that at some point in the future, Sony would bring Sophy into GT Racing 7, which is due out in March. When Sophy is able to learn more about the environment and conditions on the track, and judge the level of other drivers, an AI with this kind of intelligence and poise will be able to provide players with more real-world joy when racing against humans.
In the virtual racing game gradually "small circle", many manufacturers do not do a good job of facing the pure new players of the introductory experience today, perhaps the existence of an AI teacher, there is a chance to be able to give the virtual world of the virtual world of the virtual driving to bring more fun, as the GT Racing 4 promo title said. "Experience the automotive life."
That's probably the most important thing a game-based AI can bring to players - as Kazunori Yamauchi commented on the Sophy project, "We're not making an AI to beat humans. -We are pursuing AI to ultimately understand humans better."
Sony AI beats top human players on Nature cover3Sony said on Wednesday it had created an artificial intelligence (AI) agent called "GT Sophy" that could beat "GT Racing," a game on the PlayStation platform, Reuters reported on Feb. 9 in London. -the simulation racing game on the PlayStation platform - the world's best driver.
In a statement, the company said that in order to prepare "GT Sophie" for the game, different divisions at Sony provided basic AI research, a hyper-realistic real-world racing simulator, and the infrastructure needed for large-scale AI training.
The AI first took on four of GT Racing's top drivers in July of last year, and it learned from that race and beat the human drivers in another race that October, according to the report.
The AI's design team leader, Peter Waldman, head of Sony Artificial Intelligence America, said, "It took us about 20 PlayStation consoles, running simultaneously for about 10 to 12 days, to train 'GT Sophie' from the ground up to to reach superhuman levels."
The report noted that while AI has beaten humans in chess, mahjong, and Go, Sony said the difficulty in mastering race car driving is that many decisions have to be made in real time.
Sony's rival Microsoft reportedly recently bought Activision Blizzard for nearly $69 billion. Microsoft has been using games to improve AI in a way that continually provides new challenges for AI models.
GT Racing is a simulation racing video game that came out in 1997 and has sold more than 80 million units, according to the report.
Sony wants to apply the learnings to other PlayStation games. The company said, "There are a lot of games that can pose different challenges to AI, and we're looking forward to starting to address them."