Connect the Dots: Intel Nervana Neural Network Processor Arrives

Create: 11/21/2017 - 14:59
 Intel Nervana Neural Network Processor

Image source: Intel Corp.

Artificial intelligence (AI) is already pervasive, but IDC put a number on it: cognitive and AI technology will be a $46 billion industry by 2020. From cancer-treatment strategies to shopping ideas to predictive maintenance, AI and cognitive analytics require three things: data, analysis, and of course, efficient processing hardware.

To that end, and after showing it was serious about AI and deep learning by buying Nervana for $350 million, Intel recently unveiled the Intel® Nervana™ Neural Network Processor (NNP). With availability before year’s end, the company’s CEO, Brian Krzanich, predicts it will “revolutionize” areas such as healthcare, automotive and social media. Intel is already working with Facebook for that company’s technical insights.

According to Naveen Rao, CEO and co-founder of Nervana, the Intel® Nervana™ NNP overcomes the limitations of current processor architectures when it comes to AI. He highlights three key areas: memory architecture, interconnects and computation efficiency. 

For memory, the NNP eschews the classic cache hierarchy with Level 1 or Level 2 cache next to the processor for storage of frequently used or pre-fetched data. Instead, the NNP operates from the point of view that in-coming data is not known a priori, which is typical of IoT data. So, instead of layers of hardware for a known and expected data type and sequence, the NNP performs the management of on-chip memory resources in software directly. This provides greater flexibility, leading to better use of available silicon and faster learning times for deep learning models.

For interconnects, the processor uses on- and off-chip interconnects with high-speed bi-directional data-transfer capability. The goal of the interconnect structure is to allow neural network parameters to be distributed across multiple chips.

The approach sounds similar to typical multicore, multi-thread approaches used with current embedded systems to achieve some level of parallelism. However, those approaches get less efficient after 16 cores. How the NNP addresses the scalabilty issue in the context of neural-network parameters is as yet unclear, but according to Rao it does allow multiple chips to act as one. The net effect is that the NNP can accommodate larger models, enabling more insight.

The third aspect, Flexpoint, is a numeric format that allows scalar computations to be implemented as fixed-point multiplications and additions, while allowing for larger dynamic range using a fixed exponent. This allows each circuit to be smaller, giving more opportunity for increased parallelism on a die, at lower power.

The sum features of the Intel Nervana NNP point to potentially large gains in deep-learning training performance. Intel itself expects a 100x increase in performance in this area by 2020. While the potential is large, the next trick is to develop the algorithms to make optimal use of NNP and AI in general. There have been some hiccups along the way. 

IBM Watson Case Shows Data Ingestion and Rule-making can be Tedious

One of the issues facing AI is the ingestion of the right data in the right context, supported by the right rules for the right application. Take the medical field and IBM Watson, for example. Watson was applied to the oncology field with the aim of using it to recommend the best course of treatment to doctors, globally. However, three years in, a recent report shows that the inputting of data for such complex endeavors is slow and tedious. It also indicates that Watson is capable, at most, of reciting current literature.

This is indicative of early-stage development for AI, and one of the interviewees put it best:

“Artificial intelligence will be adopted in all medical fields in the future,” said Dr. Uhn Lee, who runs the Watson program at Gachon University Gil Medical Center in South Korea. “If that trend, that change is inevitable, then why don’t we just start early?”

To be sure, healthcare is an extreme example of the difficulties facing AI, one of which was identified in the article as being the ingestion of sufficient amounts of data from enough patients to get reliable outcomes from the analysis.

Stepping back, however, the effect of AI-like analysis is already being widely felt, albeit in under more benign yet useful circumstances, ranging from presenting options when shopping online, to speech recognition, personal assistants and personal financial analysts, aka, robo-advisors.

Between these and an oncologically prescient application lies much hard work on algorithms and data ingestion to support rapidly accelerating hardware capabilities such as the NNP, and others to come.

About Author

Patrick Mannion's picture
Patrick Mannion
Patrick Mannion is an independent writer and content consultant who has been working in, studying, and writing about engineering and technology for over 25 years.

Latest Videos