All posts in “ARM”

Arm chips with Nvidia AI could change the Internet of Things

Nvidia and Arm today announced a partnership that’s aimed at making it easier for chip makers to incorporate deep learning capabilities into next-generation consumer gadgets, mobile devices and Internet of Things objects. Mostly, thanks to this partnership, artificial intelligence could be coming to doorbell cams or smart speakers soon.

Arm intends to integrate Nvidia’s open-source Deep Learning Accelerator (NVDLA) architecture into its just-announced Project Trillium platform. Nvidia says this should help IoT chip makers incorporate AI into their products.

“Accelerating AI at the edge is critical in enabling Arm’s vision of connecting a trillion IoT devices,” said Rene Haas, EVP, and president of the IP Group, at Arm. “Today we are one step closer to that vision by incorporating NVDLA into the Arm Project Trillium platform, as our entire ecosystem will immediately benefit from the expertise and capabilities our two companies bring in AI and IoT.”

Announced last month, Arm’s Project Trillium is a series of scalable processors designed for machine learning and neural networks. NVDLA open-source nature allows Arm to offer a suite of developers tools on its new platform. Together, with Arm’s scalable chip platforms and Nvidia’s developer’s tools, the two companies feel they’re offering a solution that could result in billions of IoT, mobile and consumers electronic devices gaining access to deep learning.

Deepu Tallam, VP and GM of Autonomous Machines at Nvidia, explained it best with this analogy: “NVDLA is like providing all the ingredients for somebody to make it a dish including the instructions. With Arm [this partnership] is basically like a microwave dish.”

Your next phone may have an ARM machine learning processor

ARM doesn’t build any chips itself, but its designs are at the core of virtually every CPU in modern smartphones, cameras and IoT devices. So far, the company’s partners have shipped more than 125 billion ARM-based chips. After moving into GPUs in recent years, the company today announced that it will now offer its partners machine learning and dedicated object detection processors. Project Trillium, as the overall project is called, is meant to make ARM’s machine learning (ML) chips the de facto standard for the machine learning platform for mobile and IoT.

For this first launch, ARM is launching both an ML processor for general AI workloads and a next-generation object detection chip that specializes in detecting faces, people and their gestures, etc. in videos that can be as high-res as full HD and running at 60 frames per second. This is actually ARM’s second-generation object detection chip. The first generation ran in Hive’s smart security camera.

As ARM fellow and general manager for machine learning Jem Davies, and Rene Haas, the company’s president of its IP Products Group, told me, the company decided to start building these chips from scratch. “We could have produced things on what we already had, but decided we needed a new design,” Davies told me. “Many of our market segments are power constrained, so we needed that new design to be power efficient.” The team could have looked at its existing GPU architecture and expanded on that, but Davies noted that, for the most part, GPUs aren’t great at managing their memory budget, and machine learning workloads often rely on efficiently moving data in and out of memory.

ARM stresses these new machine learning chips are meant for running machine learning models at the edge (and not for training them). The promise is that they will be highly efficient (the promise is 3 teraops per watt) but still offer a mobile performance of 4.6 teraops — and the company expects that number to go up with additional optimizations. Finding the right balance between power and battery life is at the heart of much of what ARM does, of course, and Davies and Haas believe that the team found the right mix here.

ARM expects that many OEMs will use both the object detection and ML chips together. The object detection chip could be used for a first pass, for example, to detect faces or objects in an image and then pass the information of where these are on to the ML chip, which can then do the actual face or image recognition.

“OEMs have ideas, they have prototype applications and they are just waiting for us to provide that performance to them,” Davies said.

ARMs canonical example for this is an intelligent augmented reality scuba mask (Davies is a certified diver, in case you were wondering). This mask could tell you which fish you are seeing as you are bobbing in the warm waters of Kauai, for example. But the more realistic scenario is probably an IoT solution that uses video to watch over a busy intersection where you want to know if roads are blocked or whether it’s time to empty a given trash can that seems to be getting a lot of use lately.

“The idea here to note is that this is fairly sophisticated work that’s all taking place locally,” Haas said, and added that while there is a fair amount of buzz around devices that can make decisions, those decisions are often being made in the cloud, not locally. ARM thinks that there are plenty of use cases for machine learning at the edge, be that on a phone, in an IoT device or in a car.

Indeed, Haas and Davies expect that we’ll see quite a few of these chips in cars going forward. While the likes of Nvidia are putting supercomputers into cars to power autonomous driving, ARM believes its chips are great for doing object detection in a smart mirror, for example, where there are heat and space constraints. At another end of the spectrum, ARM is also marketing these chips to display manufacturers that want to be able to tune videos and make them look better based on an analysis of what’s happening on the screen.

“We believe this is genuinely going to unleash a whole bunch of capabilities,” said Haas.

We’ve recently seen a number of smartphone manufacturers build their own AI chips. That includes Google’s Pixel Visual Core for working with images, the iPhone X’s Neural Engine and the likes of Huawei’s Kirin 970. For the most part, those are all home-built chips. ARM, of course, wants a piece of this business.

For developers, ARM will offer all the necessary libraries to make use of these chips and work with existing machine learning frameworks to make them compatible with these processors. “We are not planning to replace the frameworks but plug our IP (intellectual property) into them,” said Davies.

The current plan is to release the ML processor design to partners by the middle of the year. It should arrive in the first consumer devices roughly nine months after that.

Featured Image: Chris Ratcliffe/Bloomberg/Getty Images

Kernel panic! What are Meltdown and Spectre, the bugs affecting nearly every computer and device?

If you’re confused by the avalanche of early reports, denials, and conflicting statements about the massive security issues announced today, don’t worry — you’re far from the only one. Here’s what you need to know about Meltdown and Spectre, the two huge bugs that affect practically every computer and device out there.

What are these flaws?

Short answer: Bugs at a fundamental level that allow critical information stored deep inside computer systems to be exposed.

Security researchers released official documentation — complete with nicknames and logos —  of two major flaws found in nearly all modern central processing units, or CPUs.

It’s not a physical problem with the CPUs themselves, or a plain software bug you might find in an application like Word or Chrome. It’s in between, at the level of the processors’ “architectures,” the way all the millions of transistors and logic units work together to carry out instructions.

In modern architectures, there are inviolable spaces where data passes through in raw, unencrypted form, such as inside the kernel, the most central software unit in the architecture, or in system memory carefully set aside from other applications. This data has powerful protections to prevent it from being interfered with or even observed by other processes and applications.

Meltdown and Spectre are two techniques researchers have discovered that circumvent those protections, exposing nearly any data the computer processes, such as passwords, proprietary information, or encrypted communications.

Meltdown affects Intel processors, and works by breaking through the barrier that prevents applications from accessing arbitrary locations in kernel memory. Segregating and protecting memory spaces prevents applications from accidentally interfering with one another’s data, or malicious software from being able to see and modify it at will. Meltdown makes this fundamental process fundamentally unreliable.

Spectre affects Intel, AMD, and ARM processors, broadening its reach to include mobile phones, embedded devices, and pretty much anything with a chip in it. Which, of course, is everything from thermostats to baby monitors now.

It works differently from Meltdown; Spectre essentially tricks applications into accidentally disclosing information that would normally be inaccessible, safe inside their protected memory area. This is a trickier one to pull off, but because it’s based on an established practice in multiple chip architectures, it’s going to be even trickier to fix.

Who is affected?

Short answer: Pretty much everybody.

Chips going back to 2011 were tested and found vulnerable, and theoretically it could affect processors as far back as those released in 1995. One would hope there aren’t too many of those in use, but we may be unpleasantly surprised on that count.

Because Meltdown and Spectre are flaws at the architecture level, it doesn’t matter whether a computer or device is running Windows, OS X, Android, or something else — all software platforms are equally vulnerable.

A huge variety of devices, from laptops to smartphones to servers, are therefore theoretically affected. The assumption going forward should be that any untested device should be considered vulnerable.

Not only that, but Meltdown in particular could conceivably be applied to and across cloud platforms, where huge numbers of networked computers routinely share and transfer data among thousands or millions of users and instances.

The good news is that the attack is easiest to perform by code being run by the machine itself — it’s not easy to pull this off remotely. So there’s that, at least.

Can this be fixed?

Short answer: Only partially, and it’s going to take a while.

Many, many devices are “affected” or “vulnerable” to these flaws, but that’s not the same thing as saying they’re totally open to attack. Intel, AMD, ARM and others have had months to create workarounds and “mitigations,” which is a polite way of saying “band-aids.”

Meltdown can be fixed essentially by building a stronger wall around the kernel; the technical term is “kernel page table isolation.” This solves the issue, but there’s a cost. Modern CPU architectures assume certain things about the way the kernel works and is accessed, and changing those things means that they won’t be able to operate at full capacity.

The Meltdown fix may reduce the performance of Intel chips by as little as 5 percent or as much as 30 — but there will be some hit. Whatever it is, it’s better than the alternative.

Spectre, on the other hand, is not likely to be fully fixed any time soon. The fact is that the practice that leads to this attack being possible is so hard-wired into processors that the researchers couldn’t find any way to totally avoid it. They list a few suggestions, but conclude:

While the stop-gap countermeasures described in the previous section may help limit practical exploits in the short term, there is currently no way to know whether a particular code construction is, or is not, safe across today’s processors – much less future designs.

What will actually happen is hard to say, but there will likely be a flurry of updates that carry out various software hacks to protect against the most obvious and damaging attacks. Microsoft has already issued one for Windows; ARM has a set of mitigations for its affected chips; Amazon is updating its many servers.

How broadly and quickly will these mitigation patches be applied, though? How many devices are out there, vulnerable, right now? These updates may not be pretty, perhaps requiring changes that will break other software, drivers, and components. And all will likely involve degrading performance.

A more permanent fix will require significant changes across the board — the circuit board, that is. Basic architecture choices that have been baked into our devices for years, even decades, will have to be rethought. It won’t be easy, and it won’t be fun.

In the meantime companies are working at full capacity to minimize the apparent threat: “mitigations” that may or may not prevent some or all of the variant attacks. As usual, these patches will likely reach only a small subset of new, fast-updating users and devices, or those the company can update directly on its own. We will only know the efficacy of these measures by their performance in the real world.

It’s worth noting that there won’t be a “recall.” If this flaw affected a single device, like the battery problems in Samsung’s phones a while back, a recall would make sense. But this is an issue that affects millions, perhaps billions of devices. A recall isn’t an option.

Why are we only just hearing about this?

Short answer: A planned joint disclosure was preempted by reporters.

It’s always a bit odd to hear that companies were informed of a major security flaw like this one months ago, as was the case with Meltdown and Spectre. This particular exploit has been under investigation for some time by researchers, and word of it trickled out in the form of small updates to various operating systems addressing a hitherto-undocumented security flaw.

If the researchers just tweeted out the details when they discovered them, it would essentially be giving attackers access to that information at the same time as the companies that can fix the problem. Generally security investigators do what’s called responsible disclosure, contacting affected companies secretly, either as a simple courtesy or in order to collaborate on a solution.

In this case Google contacted Intel several months ago, and no doubt others knew to some degree as well, since Microsoft issued patches to insiders well ahead of the public announcement, and Linux distributions were likewise addressing the issue even though the papers describing the flaw were not out yet.

The plan would normally be that the affected company or companies would come up with a solution, quietly apply it, then announce both the flaw and the solution at the same time. And in fact that seems to be what was planned in this case.

But smart reporting by The Register, which among others put together the disparate pieces, seems to have forced the hands of several billion-dollar companies. The companies scrambled to finalize their statements, addressing “inaccurate” media reports and hastily issuing patches and explanations that likely weren’t due until next week.

While some may suggest that El Reg should have let things take their course, there’s a great deal to be said for not allowing the billion-dollar companies in question to completely control the narrative around a major issue like this. If the only version of the story we ever heard was one approved by their joint committee, things would likely have been painted in a different light.

As the researchers put it at the end of the the Spectre paper:

The vulnerabilities in this paper, as well as many others, arise from a longstanding focus in the technology industry on maximizing performance. As a result, processors, compilers, device drivers, operating systems, and numerous other critical components have evolved compounding layers of complex optimizations that introduce security risks. As the costs of insecurity rise, these design choices need to be revisited, and in many cases alternate implementations optimized for security will be required.

ARM’s next-gen chip design puts the focus on artificial intelligence

ARM tipped its hand today with the announcement of DynamIQ, a new technology it says will lay the groundwork for its next generation of mobile processors. Like other mobile chip makers, the company’s got a lot to contend with when it comes to future-proofing its offerings, and certainly ARM’s making some pretty big claims for what it’s calling its “biggest micro-architectural shift since […] 2011”

Central to the company’s speed boasts are its focus on future artificial intelligence, an aspect of technology that will continue to grow more central to mobile computing over the next several years, both through the proliferation of smart-assistants, autonomous vehicles and beyond.

The chipmaker certainly isn’t being modest in its AI claims, with a stated 50x boost in performance for the technology over the next three to five years, a number it says is potentially “conservative […] as its only building out projections based on AI algorithms they know about or have access to.”

Nor is ARM understated in its planned ubiquity for the technology. As with offerings from other mobile chip makers, the company is targeting a wide range of different computing platforms that move well beyond mobile. And certainly it’s well positioned to deliver on that front, having already proven itself a versatile component maker during the explosion of IoT devices over the past several years.

The company is positioning DynamIQ chips for cars (accounting for the added workload of autonomous vehicles) and connected home devices, in addition to smartphones and the like. Microsoft has already laid some of the groundwork for additional applications back in December when it announced that it would be bringing its apps to the company’s mobile processors, in an attempt to get hardware makers to build a wider variety of devices for the operating system.

Redmond also gave ARM a little bit more love last week when it announced that it would allow for Windows Server OS to run on the company’s chips. That news was a bit of a preview of today’s announcement, as the DynamIQ architecture sees the company pushing even further into server/cloud computing hardware, along with newfound networking applications.

ARM’s not giving exact dates for the technology’s anticipated arrival, only stating that it expects its hardware partners to ship an additional 100 billion ARM-based chips by the year 2021, having shipped roughly half that number between 2013 and 2017.

Superintelligent AI explains Softbank’s push to raise a $100BN Vision Fund

Anyone who’s seen Softbank CEO Masayoshi Son give a keynote speech will know he rarely sticks to the standard industry conference playbook.

And his turn on the stage at Mobile World Congress this morning was no different, with Son making like Eldon Tyrell and telling delegates about his personal belief in a looming computing Singularity that he’s convinced will see superintelligent robots arriving en masse within the next 30 years, surpassing the human population in number and brainpower.

“I totally believe this concept,” he said, of the Singularity. “In next 30 years this will become a reality.”

“If superintelligence goes inside the moving device then the world, our lifestyle dramatically changes,” he continued, pointing out that autonomous vehicles containing a superintelligent AI would become smart robots.

“There will be many kinds. Flying, swimming, big, micro, run, two legs, four legs, 100 legs,” he added, further fleshing out his vision of a robot-infested future.

Son said his personal conviction in the looming rise of billions of superintelligent robots both explains his acquisition of UK chipmaker ARM last year, and his subsequent plan to establish the world’s biggest VC fund.

“I truly believe it’s coming, that’s why I’m in a hurry – to aggregate the cash, to invest,” he noted.

Son’s intent to raise $100BN for a new fund, called the Softbank Vision Fund, was announced last October, getting early backing from Saudi Arabia’s public investment fund as one of the partners.

The fund has since pulled in additional contributors including Foxconn, Apple, Qualcomm and Oracle co-founder Larry Ellison’s family office.

But it has evidently not yet hit Son’s target of $100BN as he used his MWC keynote as a sales pitch for additional partners. “I’m looking for a partner because we alone cannot do it,” he told delegates, smiling and opening his arms in a wide gesture of appeal. “We have to do it quickly and here are all kinds of candidates for my partner.”

Son said his haste is partly down to a belief that superintelligent AIs can be used for “the goodness of humanity”, going on to suggest that only AI has the potential to address some of the greatest threats to humankind’s continued existence — be it climate change or nuclear annihilation.

Though he also said it’s important to consider whether such a technology will be “good or bad”.

“It will be so much more capable than us –- what will be our job? What will be our life? We have to ask philosophical questions,” he said. “Is it good or bad?”

“I think this superintelligence is going to be our partner. If we misuse it it’s a risk. If we use it in good spirits it will be our partner for a better life. So the future can be better predicted, people will live healthier, and so on,” he added.

Given this vision for billions of superintelligence connected devices fast-coming down the pipe, Son is unsurprisingly very concerned about security. He said he discusses this weekly with ARM engineers. And described how one of his engineers had played a game to see how many security cameras he could hack during a lunchtime while waiting for his wife. The result? 1.2M cameras potentially compromised during an idle half hour or so.

“This is how it is dangerous, this is how we should start thinking of protection of ourself,” said Son. “We have to be very very careful.

“We are shipping a lot of ARM chips but in the past those were not secure. We are enhancing very quickly the security. We need to secure all of the things in our society.”

Son also risked something of a Gerald Ratner moment when he said that all the chips ARM is currently shipping for use in connected cars are not , in fact, secure. Going so far as to show a video of a connected car being hacked and the driver being unable to control the brakes or steering.

“There are 500 ARM chips [in one car] today — and none of them are secure today btw!” said Son. (Though clearly he’s working hard with his team at ARM to change that.)

He also discussed a plan to launch 800 satellites in the next three years, positioned in a nearer Earth orbit to reduce latency and support faster connectivity, as part of a plan to help plug connectivity gaps for connected cars — describing the planned configuration of satellites as “like a cell tower” and like “fiber coming straight to the Earth from space”.

“We’re going to provide connectivity to billions of drivers from the satellites,” he said.

For carriers hungry for their next billions of subscribers as smartphone markets saturate across the world, Son painted a pictured of vast subscriber growth via the proliferation of connected objects — which handily of course also helps his bottom line, as the new parent of ARM.

“If I say number of subscribers will not grow it’s not true,” he told the conference. “Smartphones no — but IoT chips will grow to a trillion chips — so we will have 1TR subscribers in the next 20 years. And they will all be smart.”

“One of the chips in our shoes in the next 30 years will be smart than our brain. We will be less than our shoes! And we are stepping on them!” he joked. “It’s an interesting society that comes.

“All of the cities, social ecosystem infrastructure will be connected,” he added. “All those things will be connected. All connected securely and managed from the cloud.”