CNN vs DCGAN

DCGANs stand for Deep Convolutional Generative Adversarial Networks. It is quite the contrary to a Convolutional Neural Network (CNN). It works in an opposite direction compared to a CNN. What CNN does is that it transforms an image to class labels, that is a list of probabilities, whereas DCGAN generates an image from random parameters.

CNNs and DCGANs

 

Some of you might wonder what are convolutions. Convolutions are the operations or the modifications we perform on an image. We perform modifications on the image kernel by multiplying it with the matrix of the operation we want to perform. For detailed information on image kernel and convolutions please visit here.

Moving on, what CNN does is it applies a lot of filters to extract various features from a single image. CNN applies multi-layered filters to a single image to extract features moving deeper into the layers.

CNN1

Now the typical working on CNNs is that it starts from a single RGB image on the right, multiple filtering layers are applied to produce smaller and large number of images.

cnn

Flow of CNNs

 

Image generation flow of DCGANs

DCGANs

Flow of DCGANs

 

Now, the filters that we previously mentioned are convolutional in CNNs and transposed-convolutional in DCGANs, both of them work in the opposite direction.

padding_strides

Illustration of their working

Convolution: (Bottom Up) 3×3 blue pixels contribute to generating a single green pixel. Each of 3×3 blue pixels multiplied by the corresponding filter value, and the results from different blue pixels summed up to be a single green pixel.

Transposed-Convolutions: (Top Down) A single green pixel contributes to generating 3×3 blue pixels. Each green pixel is multiplied by each 3×3 filter values and the results from different green pixels are summed up to be a single blue pixel.

 

It is suggested that the input parameters could use a semantic structure as in the following example-

inputpara

Interpretation of Input Parameters

 

Training Strategies:

CNN: Classifying authentic and fake images. Here, authentic images are provided as training data to the model.

DCGAN: It is trained to generate images classified as authentic by the CNN. It works by trying to fool the CNN, DCGAN learns to generate images similar to the training data.

 

Advertisements

What is Meta-Learning in Machine Learning?

Meta-Learning is a subfield of machine learning where automatic learning algorithms are applied on meta-data. In brief, it means Learning to Learn. The main goal is to use meta-data to understand how automatic learning can become flexible in solving different kinds of learning problems, hence to improve the performance of existing learning algorithms. Which means that how effectively we can increase the learning rate of our algorithms.

Meta-Learning affects the hypothesis space for the learning algorithm by either:

  • Changing the hypothesis space of the learning algorithms (hyper-parameter tuning, feature selection)
  • Changing the way the hypothesis space is searched by the learning algorithms (learning rules)

Variations of Meta-Learning: 

  • Algorithm Learning (selection) – Select learning algorithms according to the characteristics of the instance.
  • Hyper-parameter Optimization – Select hyper-parameters for learning algorithms. The choice of the hyper-parameters influences how well you learn.
  • Ensemble Methods – Learn to learn “collectively” – Bagging, Boosting, Stacked Generalization.

Flexibility is very important because each learning algorithm is based on a set of assumptions about the data, its inductive bias. This means that it will only learn well if the bias matches the data in the learning problem. A learning algorithm may perform very well on one learning problem, but very badly on the next. From a non-expert point of view, this poses strong restrictions on the use of machine learning or data mining techniques, since the relationship between the learning problem (often some kind of database) and the effectiveness of different learning algorithms is not yet understood.

By using different kinds of meta-data, like properties of the learning problem, algorithm properties (like performance measures), or patterns previously derived from the data, it is possible to select, alter or combine different learning algorithms to effectively solve a given learning problem. Critiques of meta-learning approaches bear a strong resemblance to the critique of metaheuristic, which can be said to be a related problem.

Metalearning may be the most ambitious but also the most rewarding goal of machine learning. There are few limits to what a good meta-learner will learn. Where appropriate it will learn to learn by analogy, by chunking, by planning, by subgoal generation, etc.

OpenAI’s Virtual Wrestling Bots

OpenAI, a firm backed by Elon Musk, has currently revealed one of it’s latest developments in the fields of Machine Learning, demonstrated using the technology of virtual sumo wrestlers.

OpenAI_wrestling

These are the bots inside the virtual world of RoboSumo controlled my machine learning. They (The Bots) taught themselves through trial and error using Reinforcement Learning, a technique inspired by the way animals learn through feedback. It has proved useful for training computers to play games and to control robots. The virtual wrestlers might look slightly ridiculous, but they are using a very clever approach to learning in a fast-changing environment while dealing with an opponent. This game and it’s virtual world were created at OpenAI to show how forcing AI systems to compete can spur them to become more intelligent.

However, one of the disadvantages of reinforcement learning is that doesn’t work well in realistic situations, or where things are more dynamic. OpenAI devised a solution to this problem by creating its own reinforcement algorithm called proximal policy optimization (PPO), which is especially well suited to changing environments.

The latest work, done in collaboration with researchers from Carnegie Mellon University and UC Berkeley, demonstrates a way for AI agents to apply what the researchers call a “meta-learning” framework. This means the agents can take what they have already learned and apply it to a new situation.

Inside the RoboSumo environment (see video above), the agents started out behaving randomly. Through thousands of iterations of trial and error, they gradually developed the ability to move—and, eventually, to fight. Through further iterations, the wrestlers developed the ability to avoid each other, and even to question their own actions. This learning happened on the fly, with the agents adapting even they wrestled each other.

Flexible learning is a very important part of human intelligence, and it will be crucial if machines are going to become capable of performing anything other than very narrow tasks in the real world. This kind of learning is very difficult to implement in machines, and the latest work is a small but significant step in that direction.

 

(sources: MitTechReview, OpenAI Blog, Wired)

NVIDIA’S CHIPS FOR COMPLETE CONTROL OF DRIVERLESS CARS

The race for autonomy in cars is ubiquitous. Top car brands are working in providing complete autonomous vehicles to their customers and the future with self-driving vehicles is inevitable. Adding the cherry to the cake, Nvidia’s recently announced chip is the latest generation of its DrivePX onboard car computers called Pegasus. The device is 13 times faster than the previous iteration, which has so far been used by the likes of Audi, Tesla, and Volvo to provide semi-autonomous driving capabilities in their vehicles.

nvidia_pegasus

Nvidia Pegasus

At the heart of this semiconductor is the mind-boggling technology of Deep Learning. “In the old world, the more powerful your engine, the smoother your ride will be,” Huang said during the announcement. “In the future, the more computational performance you have, the smoother your ride will be.”

Nvidia asserts that the device is only about the size of a license plate. But it has enough power to process data from up to 16 sensors, detect objects, find the car’s place in the world, plan a path, and control the vehicles itself. Oh, and it will also update centrally stored high-definition maps at the same time—all with some resources to spare.

The new system is designed to eventually handle up to 320 trillion operations per second (TOPS), compared to the 24tn of today’s technology. That would give it the power needed to process the masses of data produced by a vehicle’s cameras and other sensors and allow it to drive completely autonomously, Nvidia said. The first systems to be tested next year will have less processing power but will be designed to scale up with the addition of extra chips.

 

(sources: MitTechReview, NvidiaBlog)

MIT uses AI to kill video buffering

We all hate it when our video is interrupted by buffering or its resolution has turned into a pixelated mess. A group of MIT researchers believe they’ve figured out a solution to those annoyances plaguing millions of people a day.

MIT discovered a way to improve video streaming by reducing buffering times and pixelation. A new AI developed at the university’s Computer Science and Artificial Intelligence Laboratory uses machine learning to pick different algorithms depending on network conditions. In doing so, the AI, called Pensieve, has been shown to deliver a higher-quality streaming experience with less buffering than existing systems.

Instead of having a video arrive at your computer in one complete piece, sites like YouTube and Netflix break it up into smaller pieces and sends them sequentially, relying on ABR algorithms to determine which resolution each piece will play at. This is an attempt to give users a more consistent viewing experience while also saving bandwidth, but it created problems. If the connection is too slow, YouTube may temporarily lower the resolution – pixelating the video- to keep it playing. And since the video is sent in pieces, skipping ahead is impossible.

There are two types of ABR: a rate-based one that measures how fast a network can transmit data and a buffer-based one tasked with maintaining a sufficient buffer at the head of the video.  The current algorithms only consider one of these factors, but MIT’s new algorithm Pensieve uses machine learning to choose the best system based on the network condition

In experiments that tested the AI using wifi and LTE, the team found that it could stream video at the same resolution with 10 to 30 percent less rebuffering than other approaches. Additionally, users rated the video play with the AI 10 to 25 percent higher in terms of ‘quality of experience.’  The researchers, however, only tested Pensieve on a month’s worth of downloaded video and believe performance would be even higher with a number of data streaming giants YouTube and Netflix have.

As a next project, the team will be working to test Pensieve on virtual-reality (VR) video.“The bitrates you need for 4K-quality VR can easily top hundreds of megabits per second, which today’s networks simply can’t support,” Alizadeh says. “We’re excited to see what systems like Pensieve can do for things like VR. This is really just the first step in seeing what we can do.”

Pensieve was funded, in part, by the National Science Foundation and an innovation fellowship from Qualcomm.

Google’s and Nvidia’s AI Chips

Google

Google will soon launch a cloud computing service that provides exclusive access to a new kind of artificial intelligence chip designed by its own engineers. CEO Sundar Pichai revealed the new chip and service this morning in Silicon Valley during his keynote at Google I/O, the company’s annual developer conference.

GoogleChip4.jpg

This new processor is a unique creation designed to both train and execute deep neural networks—machine learning systems behind the rapid evolution of everything from image and speech recognition to automated translation to robotics. Google says it will not sell the chip directly to others. Instead, through its new cloud service, set to arrive sometime before the end of the year, any business or developer can build and operate software via the internet that taps into hundreds and perhaps thousands of these processors, all packed into Google data centers.

According to Dean, Google’s new “TPU device,” which spans four chips, can handle 180 trillion floating point operations per second, or 180 teraflops, and the company uses a new form of computer networking to connect several of these chips together, creating a “TPU pod” that provides about 11,500 teraflops of computing power. In the past, Dean said, the company’s machine translation model took about a day to train on 32 state-of-the-art CPU boards. Now, it can train in about six hours using only a portion of a pod.

Nvidia

Nvidia has released a new state-of-the-art chip that pushes the limits of machine learning, the Tesla P100 GPU. It can perform deep learning neural network tasks 12 times faster than the company’s previous top-end system (The TitanX). The P100 was a huge commitment for Nvidia, costing over $2 billion in research and development, and it sports a whopping 150 billion transistors on a single chip, making the P100 the world’s largest chip, Nvidia claims. In addition to machine learning, the P100 will work for all sorts of high-performance computing tasks — Nvidia just wants you to know it’s really good at machine learning.

dgx.png

To top off the P100’s introduction, Nvidia has packed eight of them into a crazy-powerful $129,000 supercomputer called the DGX-1. This show-horse of a machine comes ready to run, with deep-learning software preinstalled. It’s shipping first to AI researchers at MIT, Stanford, UC Berkeley, and others in June. On stage, Huang called the DGX-1 “one beast of a machine.”

The competition between these upcoming AI chips and Nvidia all points to an emerging need for simply more processing power in deep learning computing. A few years ago, GPUs took off because they cut the training time for a deep learning network from months to days. Deep learning, which had been around since at least the 1950s, suddenly had real potential with GPU power behind it. But as more companies try to integrate deep learning into their products and services, they’re only going to need faster and faster chips.

 

(via Wired, Forbes, Nvidia, The Verge)

 

A peek into NVIDIA’s self driving AI.

In a step toward making AI more accountable, Nvidia has developed a neural network for autonomous driving that highlights what it’s focusing on.

Sorta follow-up to my previous blog post: Is AI Mysterious?. Some of the most powerful machine-learning techniques available result in software that is almost completely opaque, even to the engineers that build it.

nvidia_car.jpg

We still don’t know exactly how AI works, the people at Nvidia still don’t know why the AI they built wasn’t following a single instruction given by its creators. Instead, it relied entirely on an algorithm that had taught itself to drive by watching a human do it.

It simply matches input from several video cameras to the behavior of a human driver and figures out for itself how it should drive. The only catch is that the system is so complex that it’s difficult to untangle how it actually works.

Nvidia is now working to open the black box by developing a way to visually highlight what the system is paying attention to. Explained in a recently published paper, the neural network architecture developed by Nvidia’s researchers is designed so that it can highlight the areas of a video picture that contribute most strongly to the behavior of the car’s deep neural network. Remarkably, the results show that the network is focusing on the edges of roads, lane markings, and parked cars—just the sort of things that a good human driver would want to pay attention to.

PilotNet.png

Nvidia’s Convolutional Neural Network architecture for Self-Driving AI.

“What’s revolutionary about this is that we never directly told the network to care about these things,” Urs Muller, Nvidia’s chief architect for self-driving cars, wrote in a blog post.

Fascinatingly enough, this compares a lot to human intelligence where we don’t actually know how to describe certain actions we do, but we just do them.

This certainly highlights the fact that in the near future we might start seeing AI those are just like us or even Superintelligent. I highly recommend reading the book Superintelligence: Paths, Dangers, Strategies by Nick Bostrom.

(Sources: Nvidia, MitTechReview, Nvidia Blog)

Is A.I. Mysterious?

Contemplating on the book I recently began, Superintelligence: Paths, Dangers, Strategies by Nick Bostrom, the author clearly describes the future of AI as tremendously frightening and tells us in what ways AI will overpower human beings, until a point in the future where the humans will no longer be needed. Such kind of intelligence will be superior to human beings, hence coined the term Superintelligence. Nick, further mentioned in the initial pages of the book that such kind of invention would be the last invention humans would ever make, and the AI would then invent new things itself, without feeling the need of a human being.

Does AI possess a darker side? (Enter article by MitTechReview)

Last year, a strange self-driving car was released onto the quiet roads of Monmouth County, New Jersey. This experimental vehicle was developed by researchers at Nvidia, didn’t look different from other autonomous cars, but it was unlike anything demonstrated by Google, Tesla, or General Motors, and it showed the rising power of artificial intelligence. The car didn’t follow a single instruction provided by an engineer or programmer. Instead, it relied entirely on an algorithm that had taught itself to drive by watching a human do it. Getting a car to drive this way was an impressive feat. But it’s also a bit unsettling since it isn’t completely clear how the car makes its decisions. Information from the vehicle’s sensors goes straight into a huge network of artificial neurons that process the data and then deliver the commands required to operate the steering wheel, the brakes, and other systems. The result seems to match the responses you’d expect from a human driver. But what if one day it did something unexpected—crashed into a tree, or sat at a green light? As things stand now, it might be difficult to find out why. The system is so complicated that even the engineers who designed it may struggle to isolate the reason for any single action. And you can’t ask it: there is no obvious way to design such a system so that it could always explain why it did what it did.

Now, once again, such kind of experiments and research may seem obscure at the moment or may even be neglected by some people, but what if an entire fleet of vehicles start working in the manner they learned and ignoring the commands by a human.

Enter OpenAI: Discovering an enacting the path to safe artificial general intelligence.

OpenAI is a non-profit artificial intelligence (AI) research company, associated with business magnate Elon Musk, that aims to carefully promote and develop friendly AI in such a way as to benefit humanity as a whole.

Hopefully, OpenAI will help us have friendly versions of AI.

 

deepdream1

The artist Adam Ferriss created this image, and the one below, using Google Deep Dream, a program that adjusts an image to stimulate the pattern recognition capabilities of a deep neural network. The pictures were produced using a mid-level layer of the neural network.

This image sure seems kind of spooky. But who knows, there might be some hidden sarcasm in the AI that we are yet to discover.

Meanwhile, you can give it a try https://deepdreamgenerator.com/ .

 

 

Sources (MitTechReview, OpenAI, Wiki)

</ End Blog>

Artificial Intelligence ready ARM CPUs (DynamIQ)

ARM processors are ubiquitous and many of the tech gadgets we used are powered by them, furthermore, the company is showing off its plans for the future with DynamIQ. Aimed squarely at pushing the artificial intelligence and machine learning systems we’re expecting to see in cars, phones, gaming consoles and everything else, it’s what the company claims is an evolution on the existing “big.Little” technology.

arm.jpg

Here’s a high-level look at some of the new features, capabilities and benefits DynamIQ will bring to new Cortex-A processors later this year:

  • New dedicated processor instructions for ML and AI: Cortex-A processors designed for DynamIQ technology can be optimized to deliver up to a 50x boost in AI performance over the next 3-5 years relative to Cortex-A73-based systems today and up to 10x faster response between CPU and specialized accelerator hardware on the SoC that can unleash substantially better-combined performance.

arm-dynamiq2.jpg

  • Increased multi-core flexibility: SoC designers can scale up to eight cores in a single cluster and each core can have different performance and power characteristics. These advanced capabilities enable faster responsiveness to ML and AI applications. A redesigned memory subsystem enables both faster data access and enhance power management
  • More performance within restricted thermal budgets: Efficient and much faster switching of software tasks to match the right-sized processor for optimal performance and power is further enhanced through independent frequency control of individual processors
  • Safer autonomous systems: DynamIQ brings greater levels of responsiveness for ADAS solution and increased safety capabilities which will enable partners to build ASIL-D compliant systems for safe operation under failure conditions.

 

 

(source: ARM community, Engadget)

Machine Learning Speeds Up

Cloudera and Intel are jointly speeding up Machine Learning, with the help of Intel’s new Math Kernel. Benchmarks demonstrate the combined offering can advance machine learning performance over large data sets in less time and with less hardware.  This helps organizations accelerate their investments in next generation predictive analytics.

Cloudera is the leader in Apache Spark development, training, and services. Apache Spark is advancing the art of machine learning on distributed systems with familiar tools that deliver at impressive scale. By joining forces, Cloudera and Intel are furthering a joint mission of excellence in big data management in the pursuit of better outcomes by making machine learning smarter and easier to implement.

intcloud.jpg

Predictive Maintenance

By combining Spark, Intel MKL libraries, and Intel’s optimized CPU architecture machine learning workloads can scale quickly. As machine learning solutions get access to more data they can provide better accuracy in delivering predictive maintenance, recommendation engines, proactive health care and monitoring, and risk and fraud detection.

“There’s a growing urgency to implement richer machine learning models to explore and solve the most pressing business problems and to impact society in a more meaningful way,” said Amr Awadallah, chief technical officer of Cloudera. “Already among our user base, machine learning is an increasingly common practice. In fact, in a recent adoption survey over 30% of respondents indicated they are leveraging Spark for machine learning.

 

(via – Technative.io)