p2.16xlarge

Iraklis ran Hashcat (a popular password "recovery" tool) with GPU support against the largest instance size available... the p2.16xlarge.

Iraklis ran Hashcat (a popular password "recovery" tool) with GPU support against the largest instance size available... the p2.16xlarge.

MATLAB users moving their analytics and simulation workloads onto the AWS cloud require their analyses to be processed quickly.

Engineers utilizing the AWS cloud for simulations will definitely appreciate the added computational horsepower.

They have chips dedicated to video encoding/decoding with the latest cards doing h265. Nvidia has an ffmpeg patch here

By giving these cloud-based engineers access to GPUs, their computations should speed up considerably.

This depends on if you're doing live transcoding or VOD transcoding.I work with live transcode, and it can be beneficial to run on g2 instances. Running c4.4xlarge I can transcode a good number of 1080@30 in with 1080/720/480/360/240@30 out. With a proper g2 instance I can transcode more simultaneously.So cost efficiency really depends on sustained traffic levels. I scale out currently using haproxy and custom code that monitors my pool and scales appropriately. But I monitor sustained traffic levels to know when it makes financial sense to scale up.If your main concern is transcode speed, CPU is likely sufficient -- I am unable to transcode faster than real time with live transcode.

Iraklis ran Hashcat (a popular password "recovery" tool) with GPU support against the largest instance size available... the p2.16xlarge.

> I am unable to transcode faster than real time with live transcodeWell, yeah. Maybe quantum computing will change that one day!What is it you're working on? Colour me intrigued.

Anyone knows if video transcoding on GPUs (with FFMPEG) is viable nowadays? If yes, what are the gains?

From 0.90-14.40 USD/hr.

Yess thank you, thank you, thank you. I was just signing up for the Azure N Series preview but we're good to go now :).

Pricing?

Nvidia has been doing a lot of work on hardware virtualization of GPUs. A goal of which is to run dozens of low-requirements desktops (think users running MS Office in separate Windows VMs) off of a single high-end server with a single GPU.

That's not mentioned in the article, did you work on this?Disclosure: I work on Google Cloud.

Can't speak to what happens on a Windows box, but across AWS instances it's definitely the case.

At the sort of scale these instances run I'd be surprised if you're sharing any physical hardware at all, my guess is that they provision a single VM on each GPU instance.

> The GPU memory gets wiped on initialisationWell, an attempt is made. As we can see in the case of WebGL, there are a multitude of interesting effects.

It is interesting to compare this to NVidia's DGX-1 system. That server is based on the new Tesla P100 and uses NVLink rather than PCIe (about 10x faster). It boasts about 170 Tflops vs the p2.16xlarge's 64 Tflops. If you run the p2.16xlarge full time for a year it would cost about the same as buying a DGX-1. Presumably Amazon releases their GPU instances on older hardware for cost savings.

The GPUs aren't shared between VMs, each GPU is dedicated to the VM it is allocated to. The GPU memory gets wiped on initialisation, just like host memory, so there's no leakage between successive VMs allocated the same hardware.If you do absolutely care about not being multitennanted, you can stipulate exclusive hw allocation when requesting the VM.

Stupid question: are GPU safe to be shared by two tenants in a datacenter? I read previously that there are very few security mechanisms in GPU, in particular that the memory is full of interesting garbage. So I would assume no hardware enforced separation between the VMs too.

Sounds like a great way to build a custom render farm. My home comouter has a dirt cheap GPU but it works well enough for basic modeling & animation. Terrible for rendering, though. I've been thinking of using ECS to build a cluster of renderers for Maya that I can spin up when needed and scale to the appropriate size. I don't know for certain if it's cheaper than going with a service, but it sounds like it is(render farm subscriptions cost hundreds), and I would get complete control over the software being used. I am glad to hear that Amazon is doing this. Granted, I'm more of a hobbyist in this arena, so maybe it wouldn't work for someone more serious about creating graphics.

Also, doesn't include electricity, cooling, data center space, all of which might be significant?

50K$ server at maximum.

I can start g2 but not p2 instances (limit 0).

One thing I discovered recently is that for GPU machines your initial limit on AWS is 0 (meaning you have to ask support before you start one for yourself)

I think this is the right talk by him, but there's one where he implies some differences. For instance, that because they can promise exact temperature ranges they can clock them differently.

Get an GTX 1080 for your local box and use it to write and test the model, then run in cloud for training.

For anyone who interested in ML/DL on cloud.

This is great - we'll try to get our Tensorflow and Caffe AMI repo updated soon:

I tried using google cloud (switching from AWS) a few months ago and it was the worst user experience I've ever had trying to use any software / dev environment.

AWS offers a far far better experience and cost effectiveness.

MeteoSwiss uses a 12 node cluster with 16 GPUs each to compute its weather forecasts (Piz Escha/Kesch).

By the way, I own a 1080 since two weeks and I can't overstate how powerful this thing is. Even if you are not into gaming, getting a 1080 is still a considerable option if you want to experiment with deep learning.

if you factor in the cost of keeping a ML team waiting for the network to finish training, then it might be cheaper to invest $7000 and have it run in a few hours instead of weeks.

You can set up the volume a CPU only machine (even on a free instance), and then just launch that volume with a machine with these big expensive GPU's on it.

$0.9 per K80 GPU per hour, while expensive, opens up so many opportunities - especially when you can get a properly connected machine.

Still seems much better to buy your own gtx 1080 for $700 which you would have spent in a month playing with parameters on these instances.

Using spot pricing you tend to get it far cheaper than that.

The P2 instances were designed for "heavier GPU compute workloads" such as high-performance computing (HPC), artificial intelligence (AI) and Big Data processing.

Compared to the g2.8xlarge, the largest instance in AWS' G2 instance family for accelerated computing, the P2 provides up to "seven times the computational capacity for single precision floating point calculations and 60 times more for double precision floating point calculations,"

> I am unable to transcode faster than real time with live transcodeWell, yeah. Maybe quantum computing will change that one day!What is it you're working on? Colour me intrigued.

This depends on if you're doing live transcoding or VOD transcoding.I work with live transcode, and it can be beneficial to run on g2 instances. Running c4.4xlarge I can transcode a good number of 1080@30 in with 1080/720/480/360/240@30 out. With a proper g2 instance I can transcode more simultaneously.So cost efficiency really depends on sustained traffic levels. I scale out currently using haproxy and custom code that monitors my pool and scales appropriately. But I monitor sustained traffic levels to know when it makes financial sense to scale up.If your main concern is transcode speed, CPU is likely sufficient -- I am unable to transcode faster than real time with live transcode.

From 0.90-14.40 USD/hr.

Anyone knows if video transcoding on GPUs (with FFMPEG) is viable nowadays? If yes, what are the gains?

They have chips dedicated to video encoding/decoding with the latest cards doing h265. Nvidia has an ffmpeg patch here

Pricing?

Yess thank you, thank you, thank you. I was just signing up for the Azure N Series preview but we're good to go now :).

> The GPU memory gets wiped on initialisationWell, an attempt is made. As we can see in the case of WebGL, there are a multitude of interesting effects.

Nvidia has been doing a lot of work on hardware virtualization of GPUs. A goal of which is to run dozens of low-requirements desktops (think users running MS Office in separate Windows VMs) off of a single high-end server with a single GPU.

At the sort of scale these instances run I'd be surprised if you're sharing any physical hardware at all, my guess is that they provision a single VM on each GPU instance.

That's not mentioned in the article, did you work on this?Disclosure: I work on Google Cloud.

Can't speak to what happens on a Windows box, but across AWS instances it's definitely the case.

The GPUs aren't shared between VMs, each GPU is dedicated to the VM it is allocated to. The GPU memory gets wiped on initialisation, just like host memory, so there's no leakage between successive VMs allocated the same hardware.If you do absolutely care about not being multitennanted, you can stipulate exclusive hw allocation when requesting the VM.

Stupid question: are GPU safe to be shared by two tenants in a datacenter? I read previously that there are very few security mechanisms in GPU, in particular that the memory is full of interesting garbage. So I would assume no hardware enforced separation between the VMs too.

It is interesting to compare this to NVidia's DGX-1 system. That server is based on the new Tesla P100 and uses NVLink rather than PCIe (about 10x faster). It boasts about 170 Tflops vs the p2.16xlarge's 64 Tflops. If you run the p2.16xlarge full time for a year it would cost about the same as buying a DGX-1. Presumably Amazon releases their GPU instances on older hardware for cost savings.

Also, doesn't include electricity, cooling, data center space, all of which might be significant?

50K$ server at maximum.

Sounds like a great way to build a custom render farm. My home comouter has a dirt cheap GPU but it works well enough for basic modeling & animation. Terrible for rendering, though. I've been thinking of using ECS to build a cluster of renderers for Maya that I can spin up when needed and scale to the appropriate size. I don't know for certain if it's cheaper than going with a service, but it sounds like it is(render farm subscriptions cost hundreds), and I would get complete control over the software being used. I am glad to hear that Amazon is doing this. Granted, I'm more of a hobbyist in this arena, so maybe it wouldn't work for someone more serious about creating graphics.

I think this is the right talk by him, but there's one where he implies some differences. For instance, that because they can promise exact temperature ranges they can clock them differently.

I can start g2 but not p2 instances (limit 0).

One thing I discovered recently is that for GPU machines your initial limit on AWS is 0 (meaning you have to ask support before you start one for yourself)

AWS offers a far far better experience and cost effectiveness.

This is great - we'll try to get our Tensorflow and Caffe AMI repo updated soon:

For anyone who interested in ML/DL on cloud.

I tried using google cloud (switching from AWS) a few months ago and it was the worst user experience I've ever had trying to use any software / dev environment.

MeteoSwiss uses a 12 node cluster with 16 GPUs each to compute its weather forecasts (Piz Escha/Kesch).

Get an GTX 1080 for your local box and use it to write and test the model, then run in cloud for training.

By the way, I own a 1080 since two weeks and I can't overstate how powerful this thing is. Even if you are not into gaming, getting a 1080 is still a considerable option if you want to experiment with deep learning.

if you factor in the cost of keeping a ML team waiting for the network to finish training, then it might be cheaper to invest $7000 and have it run in a few hours instead of weeks.

You can set up the volume a CPU only machine (even on a free instance), and then just launch that volume with a machine with these big expensive GPU's on it.

Still seems much better to buy your own gtx 1080 for $700 which you would have spent in a month playing with parameters on these instances.

Using spot pricing you tend to get it far cheaper than that.

$0.9 per K80 GPU per hour, while expensive, opens up so many opportunities - especially when you can get a properly connected machine.

Iraklis ran Hashcat (a popular password "recovery" tool) with GPU support against the largest instance size available... the p2.16xlarge.

Yess thank you, thank you, thank you. I was just signing up for the Azure N Series preview but we're good to go now :).

AWS has launched a new family of Elastic Compute Cloud (EC2) instance types called P2. Backed by the Tesla K80 GPU line from Nvidia, the new P2 instances were designed to chew through tough, large-scale machine learning, deep learning, computational fluid dynamics (CFD) seismic analysis, molecular modeling, genomics and computational finance workloads

Iraklis ran Hashcat (a popular password "recovery" tool) with GPU support against the largest instance size available... the p2.16xlarge.

I'm currently looking for the correct number of 2018 budget for the AWS instances in preparation of more deep neural net training, but now a bit confused because of this chart below: p2.16xlarge costs 3 times more than g3.16xlarge with the same number of processors but slightly less memory. Is this diff just because of the memory difference? or is there any other key difference between p2 and g3 instances?

This depends on if you're doing live transcoding or VOD transcoding.I work with live transcode, and it can be beneficial to run on g2 instances. Running c4.4xlarge I can transcode a good number of 1080@30 in with 1080/720/480/360/240@30 out. With a proper g2 instance I can transcode more simultaneously.So cost efficiency really depends on sustained traffic levels. I scale out currently using haproxy and custom code that monitors my pool and scales appropriately. But I monitor sustained traffic levels to know when it makes financial sense to scale up.If your main concern is transcode speed, CPU is likely sufficient -- I am unable to transcode faster than real time with live transcode.

> I am unable to transcode faster than real time with live transcodeWell, yeah. Maybe quantum computing will change that one day!What is it you're working on? Colour me intrigued.

They have chips dedicated to video encoding/decoding with the latest cards doing h265. Nvidia has an ffmpeg patch here

Anyone knows if video transcoding on GPUs (with FFMPEG) is viable nowadays? If yes, what are the gains?

From 0.90-14.40 USD/hr.

Pricing?

Nvidia has been doing a lot of work on hardware virtualization of GPUs. A goal of which is to run dozens of low-requirements desktops (think users running MS Office in separate Windows VMs) off of a single high-end server with a single GPU.

That's not mentioned in the article, did you work on this?Disclosure: I work on Google Cloud.

At the sort of scale these instances run I'd be surprised if you're sharing any physical hardware at all, my guess is that they provision a single VM on each GPU instance.

Can't speak to what happens on a Windows box, but across AWS instances it's definitely the case.

It is interesting to compare this to NVidia's DGX-1 system. That server is based on the new Tesla P100 and uses NVLink rather than PCIe (about 10x faster). It boasts about 170 Tflops vs the p2.16xlarge's 64 Tflops. If you run the p2.16xlarge full time for a year it would cost about the same as buying a DGX-1. Presumably Amazon releases their GPU instances on older hardware for cost savings.

Sounds like a great way to build a custom render farm. My home comouter has a dirt cheap GPU but it works well enough for basic modeling & animation. Terrible for rendering, though. I've been thinking of using ECS to build a cluster of renderers for Maya that I can spin up when needed and scale to the appropriate size. I don't know for certain if it's cheaper than going with a service, but it sounds like it is(render farm subscriptions cost hundreds), and I would get complete control over the software being used. I am glad to hear that Amazon is doing this. Granted, I'm more of a hobbyist in this arena, so maybe it wouldn't work for someone more serious about creating graphics.

The GPUs aren't shared between VMs, each GPU is dedicated to the VM it is allocated to. The GPU memory gets wiped on initialisation, just like host memory, so there's no leakage between successive VMs allocated the same hardware.If you do absolutely care about not being multitennanted, you can stipulate exclusive hw allocation when requesting the VM.

> The GPU memory gets wiped on initialisationWell, an attempt is made. As we can see in the case of WebGL, there are a multitude of interesting effects.

Stupid question: are GPU safe to be shared by two tenants in a datacenter? I read previously that there are very few security mechanisms in GPU, in particular that the memory is full of interesting garbage. So I would assume no hardware enforced separation between the VMs too.

Also, doesn't include electricity, cooling, data center space, all of which might be significant?

50K$ server at maximum.