This is just a fun quick post. Have you ever been traveling across the pond and wanted to watch your favorite streaming service or news broadcast. You fire up your tablet tap on the app or go to the website and you get… “we’re sorry this content is unavailable out side of the…”
Annoying message when you just want to watch your favorite shows
I have a couple of ways to address this. First I use PiVPN to have a secure tunnel back home. That will work a lot of the time… Though sometimes the lag just isn’t worth it. The other way I watch the latest episode of that new show is to change the appearance of where my device is.
To do that I use a service called Unlocator. (There are others out there to.) They have two different features, one is a VPN service, which can be handy. The other is a DNS service provider that hides your real location. Meaning if you’re like me and want to catch the latest episode of a popular show, just leverage Unlocator and change to a location where you can watch.
Unlocator costs me about $25 USD a year (I just use the DNS service). So when I travel I can get services like Netflix, Hulu, Disney+, and the big 3 national broadcasting services that are here in the States.
You might wonder how to setup Unlocator with your device. It’s actually rather easy.
Login to your Unlocator account to see what your DNS entries should be.
On your device, add those DNS entries to the top of your DNS list. (Don’t worry they have instructions on how to do it for just about any device imaginable.)
Verify Unlocator is working correctly for you by clicking the “check” button in your Unlocator account.
Pull up your streaming service
That’s it, nothing more than that. Now you can watch your favorite programs from across the pond. (Yes, it’s really that simple.) I actually have a Raspberry Pi that I have setup with all my favorite streaming services, a VPN, and Unlocator. Then I just hook it up to the TV in my hotel room and away I go.
Those of you in the States might also wonder if this means you could watch your favorite overseas content on for example the BBC. That would be wrong because you’re supposed to have a TV license to watch BBC programme and the BBC asks you when you load iPlayer if you have one… So I guess the answer is no.
In March 2019 NVIDIA released the Jetson Nano, a $99 GPU embedded micro-computer. I have several and have even distributed them to some brilliant youth to see what they come up with. I’ve noticed a couple things about the Nano’s that could have very interesting results on how we process data in organizations and smart cities. Let’s take a look.
The first thing I’ve noticed, Stratodesk has blogged about using Nano’s as thin client systems for VDI with good graphics capabilities (the Nano’s GPU contributing to that). This improves the user experience while keeping cost for desktop endpoints relatively low. Imagine hundreds of Nano thin clients through out the cubes of an organization… End users are getting a great graphics experience at a pretty economical price. Cube’s everywhere are glowing green with Jetson Nano’s as users monitors power up…
In most organizations GPU powered thin clients (or any thin clients for that matter) are used for about 8 hours a day (when users are in the office) and the rest of the time they sit idle, not delivering value to the organization. They may be on or off, just sitting there idling waiting till the next work day roles around to add value to the organization.
The second thing I’ve noticed, at GTC Silicon Valley 2018 Liqid presented session s8539 on pooling and orchestrating NVIDIA Jetson for AI and deep learning on the edge. Essentially being able to take NVIDIA Jetson’s and compose them on a high speed network in a single chassis as single unified system. You could have up to 24 Jetson’s composed as a single platform, running in your data center.
Interestingly enough, after doing some digging, this also includes the Jetson Nano as one of the Jetson types that could be composed. That creates a pretty spiffy AI/DL platform for the data center or remote location. Pop it in a rack, use the data center cooling and power, to start cranking on those ever vexing business questions (like figuring out who’s the masked singer).
Independently these are very cool options both of which can take advantage of the Jetson Nano. At this point queue the WonderNerd, and his hair brained ideas. Why not bring them together and do a little cube farming IT style? I bounced the idea of composing thin clients into a processing system off of someone I know at Liqid to see if it had wings and well lets start putting the idea together and see what you think.
The basic idea is (stop me if you’ve heard this VDI by day concept) that we have a bunch of endpoints with NVIDIA GPUs (like Jetson Nano’s) spread through the cubes of an organization. During the day they provide a VDI endpoints to users. When the users go home for the day and the endpoints are sitting idle, we recompose them and turn them into a grid based computing platform (don’t confuse this with NVIDIA GRID, that’s virtualizing GPUs, we are aggregating them). It’s a concept similar to VDI by day and compute by night, only this is end point by day and compute by night.
This isn’t necessarily a new idea. People have been doing this for quite a while. It’s called grid, or distributed, computing. I’ve been doing grid computing for quite a while, volunteering my systems in a program called World Community Grid (WCG) which has been helping researchers solve various world problems like Zika and childhood cancer. If you have some spare compute cycles to share, I recommend participating in the WCG.
The idea behind grid, or distributed computing is that you have a bunch of systems distributed throughout an area. Each system reaches out to a master node to get a bit of work to do, it processes the job and returns the result to the master node. The same bit of work would, in most circumstances, be distributed to three or more systems participating in the grid. The answer in the majority is accepted as the correct result. (Some of the foundations of block chain stem from distributed computing.)
This probably seems straight forward enough, take a bunch of Nano’s and compose them as a GRID platform. This is all well and good if that’s all these were doing, but their primary purpose is as a thin client end point so users can get work done.
You may be asking, how would thin clients change state from an endpoint to part of a composed infrastructure system? Thinking about this, it could be done by leveraging some intelligent programming to detect when thin clients were no longer in use and reallocate them to a distributed composable infrastructure. (Much like what I did with my VDI by day compute by night scripts.) With a bit more intelligence they could have some code that pays attention to usage habits and removes a thin client from a processing system 30 minutes prior to the expected arrival of the person who uses it as a thin client.
You’re probably thinking that I’m forgetting one little thing, networking, no one’s going to run 10GB lines to end users. That’s insane and kills the benefits ($$$) of something like this, besides Nano’s dont have 10Gbps links. 1GB links should work in many situations. What may need to happen is a way to add an M.2 card on the thin client to be used as a store and forward buffer (no the Nano’s don’t currently have one). The micro SD storage slot on the Nano is a consideration, but I’m not sure how durable it would be functioning as a cache. With a buffer it would be possible to fully saturate the link in both directions as material is created and transferred through out the grid. If an M.2 were available, hopefully it would sit on the same bus as the Nano’s GPU thus it should be fast enough to supply the GPU, and with the right caching algorithm minimize the impact on the network and individual nodes.
With some M.2 instances being composeable as well, it might be possible to create both a local and unified storage space on each endpoint. This would allow processing jobs to function as both dis-aggregated and aggregated processes. This might be an area where VMware vSAN would work, though network speed between Nano’s may be an issue. The data would need to be placed locally first and then propagated to other nodes thus creating unified storage among all the nodes. This might be one method to aggregate and share processed results among the nodes.
Even with local storage, this would still require the up-link’s off the network switches and the switches bus to be pretty beefy. The minimum bandwidth up to support something like this would be two – 10 Gbps ports LAG’ed together and 40 Gbps would probably be preferable.
In most cases it would also be best to limit the size of a composed system to a single switch. In other words if the switch has 24 ports, the maximum system size is 24 nodes. This helps avoid extra hops and network lag plus we don’t want to congest the north bound network links with east west traffic from nodes not on the same switch. Again this would be something that could be addressed with some intelligent programming.
With the networking out of the way, what could a Jetson Nano cluster deliver for an organization. Here are the Nano specs…
Let’s extrapolate that out to a single 24 node composed system and see what the total power of it would be…
Component
Single Jetson Nano Capacity
Combined Capacity of 24 Jetson Nano’s
GPU
128-core Maxwell
3072 Maxwell cores
CPU
Quad-core ARM A57 @ 1.43 GHz
96 ARM A57 cores @ 1.43 GHz
Memory
4 GB 64-bit LPDDR4 25.6 GB/s
96 GB LPDDR4 RAM
Connectivity
Gigabit Ethernet, M.2 Key E
24 Gbps of combined network connectivity
At the GPU level, this infrastructure would be equivalent to one NVIDIA M6000 GPU which has 3072 Maxwell cores. At the time of this writing Amazon is selling the M6000 for $1999.00 USD. Using a base cost model a single GPU would win, $1999 (1 x $1999) vs $2400 (24 x $100) for Jetson Nanos. Plus all the added complexity (switch, storage, etc.) and coding (operational state, network segment, etc.) required for a Nano farm.
Thought that’s not a true apples to apples comparison. The M6000 would be used exclusively for workload processing, a composed system with Jetson Nano’s would have a dual function, end point and workload processing. That would mean that 8 hours of the day they serve as end points and the other 16 (ish) hours they are doing processing. Now we could weight the cost of this on the hours used by each application, which would work out to $800 for thin client and $1600 for processing, thought that’s not a fair way to look at it either. The thin clients are required (assuming VDI), they are a sunk cost to the organization. In other words they are already there, they cost a given amount (for the purposes of this blog $100) regardless of if they are used for 8 hours or 24 hours.
That means a better way to approach this cost would be something like, “the data processing team pays for the upgraded switch(s) and the software to control it as a cluster along with any back end equipment needed for a composed system.” Granted this still probably wouldn’t be economical at a small scale of 24 thin-clients… this would need to be much larger deployment. Think about an office of 500 cubes, that would be 20 composed systems when no one is in the office.
500 cubicles
That’s 20 extra M6000 GPUs working about 16 hours a day on the organizations problems. That works out to roughly 278 extra processing days a year per composed system, or 2,254 days for 20 composed systems. (5 days a week X 52 weeks a year = 260 days * (2/3 of a day) = 173.333 days + 104 weekends = 278 days.) This works out to about 76% of a composed systems time could go to data processing over a given year. That’s a pretty good result.
At this time my concept probably doesn’t make complete business sense (give it time). The biggest hindrance is someone taking the time to program the logic described. Additionally I’m not sure what the performance degradation would be splitting a compute job across 24 nodes instead of running it on a single processor (there is more latency sending a signal over a TCP/IP network than a few nano-meters of copper). Because of these unknowns, I can’t say this will save you tons of money, in fact I’m not sure what the cost of this at scale would be right now especially if you calculate the cost based on usage or a direct comparison to a dedicated GPU. Still its interesting to consider in large organizations.
This thought process does open up a couple of other interesting opportunities though, where it could prove adventitious.
One is for fractional workload processing. Yes, that thing that all my AI/ML/DL/HPC folks are wanting for their workloads. Let’s say you have a bunch of small jobs that don’t consume a whole M6000 GPU, they only consume, lets say half of the GPU, but you must consume the whole card to run the job. Now lets say you have 1 million of those jobs to run. Lets say each job takes 3 seconds to run. That’s 3 million seconds to run all the jobs. Now lets say I can optimize the operation by splitting it and running it on two systems half the size of the original… that’s 1.5 million seconds to process all those jobs. In other words I’ve gone from 833.33 hours to 416.66 hours. That’s a pretty powerful way to optimize resources to fit the workload. It can also scale up. (In other words 1.5 GPUs instead of 2.)
There is also the possibility to do this programmatically. Which would mean that a function in the program could determine the optimum processing configuration and compose the infrastructure accordingly. This is much further off though.
Fractional GPU processing can be done by using composability proposed by Liqid in their GTC session mentioned above. It can also be done using virtualization technology such as VMware vSphere. My proposal with Nano’s just leverages unused resources in the organization.
Fractional GPU workload processing
The second scenario is also interesting, especially with 5G, micro services, containers, smart cities, and many more advancements. Imagine a smart city with hundreds of smart systems processing things in real time, like traffic lights, utilities, etc. Now imagine them functioning as a large unified system which would allow processing power to go where it is needed.
Think about an intersection that is busy at night with hundreds of cars traveling through it every hour. Across town there is another intersection that is only busy during rush hour. And, yet, another intersection that is only busy on the weekends. A composed infrastructure like I have described above could be used to deliver an optimized pool of resources across a smart city. The unused processing resources from one area (purple circle) could be used to enhance near by areas that need more processing power (red circle) during peak times. It becomes a dynamic proactive city that responds to the changing needs of its residents and visitors.
All of these are interesting concepts, they just need someone to build them… maybe I’ll see if I can build a composable Nano infrastructure in my home lab. If you’ve already built this please share with the readers and I, post a link to it in the comments below.
May your servers keep running and your data center always be chilled.
I can finally tell you what I’ve been working on for the last 17 months. Today Dell EMC announced the PowerOne System. This is what I’ve been working on, it hasn’t been a VDI program or anything like that. My work at Dell has been part of a team creating an autonomous, outcome oriented, converged infrastructure.
You might be asking — what is a PowerOne System? PowerOne is an autonomous converged infrastructure that is a giant leap forward in the industry and provides outcome oriented results. It is built on all Dell components. That means storage, compute, and networking are all from Dell.
The storage array for the initial release of PowerOne can be either PowerMax 2000 or PowerMax 8000
Workload computing currently runs on the Dell EMC MX7000 with either MX740c or MX840c compute sleds
Dell PowerSwitch networking provides connectivity to all of the components
At the heart of PowerOne resides the PowerOne Controller which directs automated outcome oriented results
What does “automated” and “outcome oriented” mean? It means that the components needed for workloads are brought together with automation and delivered as a declared outcome. This is a key concept that is unique to PowerOne. Let’s get even deeper on this with two examples.
Let’s say you are at a restaurant — you are given a menu of food available from the restaurant. You select your drink, appetizers, entree, and dessert. You don’t have to know how to make any of them. The chefs, bartenders, and wait staff know how to make it all for you and bring it to your table complete and with high quality. PowerOne works in much the same way: you declare the outcome you want, PowerOne then takes that and looks at what it has in inventory, and using stringent best practices, delivers the infrastructure for you to consume.
If you don’t dine out all that often, maybe cars are a better model. Consider driving down the road and you set your cruise control to the speed limit, take your foot off the gas, and go with the flow of traffic. You’ve declared an outcome: you want to go the speed limit. You don’t need to know all the complex operations for making that work, you just push a button. This is the same with PowerOne — you declare the outcome and PowerOne delivers that outcome without you having to know all of the inner workings and best practices to get the result you want.
All this declarative stuff is great, right? How does PowerOne deliver these outcome oriented results? They are delivered with the PowerOne Controller, the heart of PowerOne, that I mentioned above. The PowerOne Controller is a redundant appliance that manages all of the PowerOne infrastructure and delivers and maintains outcomes.
How does it do that? With a lot of cool programs running inside the PowerOne Controller. Remember, the PowerOne Controller is an appliance. When was the last time you bought a refrigerator and asked the sales person what sort of refrigerant it used. You probably haven’t ever asked, I just bought one last week and didn’t even bother to ask.
And you are probably thinking, well how am I supposed to be able to use PowerOne and make it fit into my environment? Great Question, I’m glad you asked! PowerOne has a fully functional API so you can leverage it as part of your data center automation tools. That means, if you want an automation process to stand up a new vSphere cluster whenever a request is approved, or have a script that watches your DevOps environment and adds a new host when resources are constrained, you can do it with PowerOne without needing to know the recipes.
Remember earlier when I talked about how outcome oriented results from PowerOne were like going to a restaurant? The restaurant doesn’t tell you how they make the chicken soup, nor do you have to tell them how to make it, you just declare “I’ll have the chicken soup.” Everything else is done behind the scenes. Same thing is happening with PowerOne, you just declare what you want and PowerOne prepares it for you.
Beyond the PowerOne API as a way to declare outcomes, PowerOne also has a UI that lets you declare outcomes. It’s called the PowerOne Navigator and is built on top of the PowerOne API. You can even try it out on the Dell Demo Center by looking for interactive demo ITD-0315 PowerOne Navigator. The PowerOne Navigator lets you engage in a declarative approach to IT infrastructure using a UI. (Seriously, go check out the demo.)
You’re probably wondering about how all of these outcomes are delivered and made available for consumption. The resources (storage, compute, and network) are logically configured together as a CRG or Cluster Resource Group. This is a logical grouping of infrastructure resources and their settings. You can have a bunch of CRGs.
Once a CRG has been declared, the hosts in it have VMware ESXi automatically installed on them and they are added to a vCenter for management. All of that is done automatically without anything more than declaring an outcome. The IT staff no longer need to know about all the obscure best practices for creating vSphere environments. PowerOne already takes those into account.
Below is the starting screen for creating a CRG. Notice, I declare a name, a version of vSphere, and whether I want to base its design off of any other CRGs I already have. A few more screens and I have a functional CRG with ESXi installed on it.
That brings up something that is really cool with PowerOne. You may have noticed here that it’s asking what version of ESXi I want installed on the CRG. PowerOne allows for multiple versions of vSphere in an environment. That means when the organization is ready to move a given workload to a new vSphere version, they can update it without affecting other workloads on other CRGs.
Does that mean each CRG has its own vCenter server instance? No, by default there are only two vCenter instances in PowerOne and they are located in a vSphere Management Cluster. One vCenter appliance is responsible for controlling the vSphere Management Cluster and the other is responsible for all of the other CRGs.
In addition, the vSphere Management Cluster holds the NSX-v components, vROps nodes, and a data protection VM. This leaves some space for additional management workloads to be installed on the vSphere Management Cluster.
There are a lot of other cool things in PowerOne I could share but this is already a fairly long post, so we’ll save those for another time. If there is something particular you’d like to know about PowerOne, let me know in the comments or DM me on twitter at @wonder_nerd.
May your servers keep running and your data center always be chilled.
A special thanks to Bob Percy for reviewing this post prior to publication.
I thought I would do a quick blog post about the enrollment of iOS 9.3 devices into VMware Workspace One (WS1). iOS 9.3 is the last release for several versions of devices. Well there are many iOS devices still out there that use iOS 9.3.x, which means if you are doing mobile device management (MDM) in your organization you may run into something that I ran into this week.
I’m going to say this up front… iOS 9.3 is supported with WS1 9.6. In fact support goes all the way back to iOS 7. (If you are on iOS 7, if at all possible consider upgrading to iOS 9 it provides a lot of great functionality.) And if you are an end user reading this blog because your IT team said your device is to old point them to the VMware documentation…
That said, here’s what I ran into. When enrolling an iOS 9.3 device with the AirWatch Agent. I would make it to step 5 in the list. At which point it would launch a safari web browser to get to the AirWatch MDM site for the organization, so that the device can download a MDM profile. Once it got to the site the progress bard would only progress to about 66% then it would stop responding. If you tried to do anything safari would crash and you would need to restart the enrollment process from the beginning.
This was on a clean iOS device that had been rebooted prior to installing the AirWatch agent. What could be wrong… After verifying it wasn’t an ID10T error with the user the next step was to clear the cache for the browser. To do this, go to settings, scroll down to Safari and tap on it, lastly scroll down to “Clear History and Website Data” and tap on that. This clears the cache and should hopefully fix any issues with stale files that haven’t aged off.
That of course didn’t work for the user. What could be the problem??? What else to try… I had the user try rebooting the iOS device after clearing the Safari cache. This time tings proceeded past step 5 and the device was able to download and install the MDM profile and finished the rest of the enrollment process.
That’s right if you run into an issue with your AirWatch enrollment try clearing the cache and then rebooting the device.
Yup, its that line from the IT Crowd… “Have you tried turning it off and back on again?”
The interesting thing about all of this is the user was me. I was enrolling an iOS device and this bit me. I thought I would share in case you’ve been bit by this as well.
May your servers keep running and your data center always be chilled.
These are the slides from my session on VDI by Day Compute By Night slides from my vBrownBag presentation. If you have questions please let me by know by posting in the comments sections or using the contact me page.
Light moves at a speed of 299,792.458 kilometers a second (km/s) or 186,282.397 miles per second (MPS) [Gibbs 1997]. What if we could use the speed of light and the vasts distances of space as a storage medium? In this post I propose the basic premise to do just that, to have storage at the speed of light.
I think it’s best to start out with what this is and what it is not.
This isn’t going to make storage faster. In fact this is a linear sequential storage platform like tape storage, it has to be read in the order it is transmitted. There is no jumping ahead or back to pickup additional bits.
Storage only flows in one direction. Because we are dealing with massive distances storage can only flow in one direction at a time. This becomes important later on.
Now lets get into the details of this proposal. Einstein proposed that the speed of light is a constant in his famous and simple formula E=MC2. In this formula C is a constant for the speed of light. The fastest moving thing known to humans.
Latency
On earth we experience latency, for example when we make a phone call half way around the world. This is caused by how fast we can transmit data from point A to point B. In my proposal latency is the storage medium. (Really it’s space.)
The simple formula for storage using the speed of light is distance (d) equals transmission time (t) multiplied by the speed of light (C) divided by 2. This of course assumes that the receiver (r) is capably of receiving data at the same speed as the transmitter, processing times at each end are 0, and as soon as data comes in it is sent back out in the opposite direction. This also doesn’t account for the addition of new data into the stream.
Formula for data transmission and storage at the speed of light.
You might be asking why is this divided by 2. This is because you must have two satellites. The one nearest the earth that transmits and one d distance away from the first satellite that receives the signal and then sends it back to original satellite. If there wasn’t the second satellite it would just be broadcasted in to space never to be seen again. This is shown in the diagram below:
Example of the Latency Storage Method. Satellites a given distance (d) away from each other are able to transmit (t) a given amount of data. While traveling from one satellite to the other it is in a state of storage. The data can only be read at Satellites Otherwise it is in a stored state.
Example of Latency for Storage
Lets walk through an example of this. Let’s say I can transmit and receive at 3 Mb a Second and I need to transmit 180Mb of data (including protocols). How far apart do the two satellites need to be? First we need to see how long we will transmit for. That works out to 60 seconds (180/3). We know that light travels at 299,792.458 km/s. We multiply by 60 and find that the distance is 17,987,547.48 km or one light minute (the distance light travels in one minute in a vacuum). We then need to divide that by two since it will be 30 seconds out and 30 seconds back. So our satellites would at a minimum need to be 8,993,773.74 km apart.
In the above scenario it would take 1 minute to get the data back once it was put in the loop. Why? Because we are using latency (and space/time) as the storage medium. Space time becomes a tape cartridge and our satellites the read heads. We can only read the data when it comes around to a read head. In this case the one closest to Earth otherwise we have to wait for the data.
Adding Data to the Stream
Now what if we want to continue adding data to this stream, how is that done? We can accomplish this by moving the satellites further apart. So lets say we wanted to add an additional 180Mb of data to the stream in the above example. How far do we need to move the satellites apart? That’s fairly simple we just need to double their distance.
The how fast we double their distance is the hard part. Since our satellite can’t reach the speed of light the amount of data we can add to the stream is dependent on how fast the satellite is moving away from the other satellite. This can be figured out…
Let’s assume our satellite is moving at 692,017.92 km/h (430,000 MPH). Which will be the speed of the Parker Solar Probe later in its journey, “making it the fastest human-made object relative to the sun” (Parker 2018). That means, if our satellite were to travel at the same speed it would be going 11,533.632 km/m. Know that we can figure out how fast we can add data to our latency based storage.
To figure this out we will restate our original equation as the distance (11,533.632 km/m) times 2. We then divide by the speed of light (17,987,547.48 km/m). That will provide us with the time. Or about an additional 0.00128240 minutes (0.0769441 seconds) of transmission time per minute. This of course assume that the speed is constant.
Formula for transit time of data over a given distance at the speed of light.
Knowing that and that we can transmit at 3 Mbps, we could add about 0.230832331 Mb to the stream every second (0.0769441 * 3 = 0.230832331).
There are obviously faster long term storage methods than this. It’s still a very interesting way to use time and space to retain data for extended periods of time.
Limitations
There are obvious limitations to this that should be called out here.
Hardware speed is not taken into account. Once a satellite revives data it must then process it and transmit it. This takes time and would increase the amount of time and reduce the amount of data stored in the time/space medium.
Because of hardware speeds a loop could never be filled to maximum capacity and each half of the loop would need to maintain a given amount of “slack” space to accommodate the time it takes for hardware to re-transmit data.
The formula’s don’t account for jitter. If a satellite were to speed up or slow down it would change the capacity of the system.
Data would need to be encrypted. Because data would be transmitted in open space and would presumably be transmitted beyond just the two satellites, data encryption methods would need to be used.
This does not take into account a reasonable transmission time. IE if it takes two weeks for data to make a round trip is that too long.
This also doesn’t account for maximum transmission distances where the rest of the cosmic noise would drown out the data transmission. (Which would necessitate the need for a checksum system being added to the protocol.)
It might be possible to gain higher amounts of data using multiple transmit frequencies, effectively increasing storage without increasing distances.
For the longest time most have seen latency as an issue or delay to be dealt with. It looks like it can be a very powerful storage method given enough space and time. If you think of the research being done to slow the speed of light (see Physicists Slow Speed of Light) it could be very interesting in the years to come.
This whole storage notion is just a random idea I had while volunteering at a Kansas 4-H SpaceTech Experience at the Kansas Cosmosphere. We had the youth doing an activity that mimicked satellites flying past Pluto. The youth could only go so fast relaying messages to the space craft. It reminded me of packets being pulled off a tape by a tape head. It made me think, why not use this as a storage medium.
I hope you enjoyed the read. May your servers keep running and your data center always be chilled.
It’s the last day of GTC 2019, for those who know me they know it’s time for my yearly wrap up of the trade show. For the folks in charge of the show it’s feedback for next year’s GTC.
For those not familiar with my wrap up report, the last day of the show I write down all my positive and negative impressions of the show. That said here are my thoughts on this year’s GTC.
This year was a great year to attend GTC. There was the Mellanox acquisition announced the week before GTC, the release of the NVIDIA Jetson Nano (for $99), a slew of other new releases, and of course lots of fresh and new knowledge. Many of these things you can read about elsewhere, so I won’t regurgitate those here.
Here are some things from the show that may have gone unnoticed by some:
The pervasive use of the NVIDIA T4 GPU for almost all types of workloads. A true multi-use GPU.
This year there were several aspects of the show I really enjoyed. The first one was the birds of a feather (BoF) and community meetups after the sessions concluded for the day. I was able to attend the NGC community meetup and the Slurm BoF. both were outstanding with lots of information to be gained and a wealth of community resources! I think the teams that put these on were surprised by the success of the events.
I wasn’t particularly fond of the wait to get into the general session… the line for K4 seating wrapped around the building, the gates didn’t open until 30 minutes prior to the keynote and the doors didn’t open till about 15 minutes before the keynote. We were out in the sun getting a tan while we waited. There were obviously more people in the K4 group than any other group there. I would hate to see what happened had it rained the day of the keynote.
Continuing that thought Once the doors were opened for the K4 groups to pour in, there was very little lighting to see at the top level to make our way to the seats. This is especially true of people coming in from the very bright outside into a very dark environment. (The whole rods and cones visualization thing.) I would have liked to see either ushers at the top of the steps all the way along the hall way of the keynote with flash lights or some other form of light that could be controlled while people are trying to quickly find their seats.
I thought the diversity of sessions was wonderful. Every area seemed to have a wide range of options to choose from. The vGPU content wasn’t just focused on VDI there were all sorts of sessions you could chose from either based on your interest or to be exposed to something new. I also think the did a good job with session descriptions this year so you could gauge it’s technical content.
I was talking with Roger (one of the event coordinators) at the end of the show like I did last year. One thing I talked with him about was how you indicate a session is targeted to more of an executive of business leader. I proposed to him adding a prefix or suffix to the session number, for example ex12345 or 98765ex. That way it becomes very simple for non-techy types to identify sessions geared toward the business case for these solutions and not the ones that are diving deep into the guts of the solutions.
Some of you may be thinking, but GTC is a techy show, not a business show. It is, but, the smart techies know that to get funding for their programs they have to get buy in from business people. If you can bring them to the show they can see the value and come to better understand the business value. This will create a synergy that can help accelerate programs in organizations.
This leads me to one of my observations at this years GTC. The number of straight laced suits to t-shirts and pony tails appears to be almost 50/50 now. That’s right you walk the hallway’s you see a lot of folks in suits, almost as many as those with t-shirts. I even proposed a fun little experiment to Roger. Mount a camera in the main hallway use some image recognition and see how many are in suits vs t-shirts to show the change in attendance.
That gets me to the show floor. This year there were a lot of cool things. The robotics section was very cool from the Boston Robotics team to the building inspection bot. The VR experience was also off the hook. I love it when the first question after putting on the VR headset was “how are you with heights?” from there I was standing atop the highest mast of a ship. Absolutely awesome!
I don’t know that the extended hours were the best thing for the vendors, as many of them were dragging by the last day (more than normal). I also spent some time at the Dell booth (I work for Dell) and during sessions the show floor seemed really slow. So again I wonder if there is value to having the show floor open during sessions.
I do like the added space for the show floor that was gained by moving the keynote off site. That made it easier to walk down the isles.and for more exciting exhibits. I also am cool that they got rid of the Wednesday night party. It didn’t seem anyone else really missed it and created a lot more opportunities for dinner with friends and business partners or community meetups.
Another thing I didn’t care for this year is the design of the bags this year. The look of them is good, but the metal rods around the top of the bag are really annoying. It made it really difficult to fold it up and stick it in my other bag so I could drop it in my room. I get the point of the metal to keep the shape and make it easier to put things into it. I’m also not a fan of the zipper running long on the main compartment, to me that looks sort of sloppy and reminds me of tool bags I buy at a discount tool store.
I’m also sad that attendees did not get t-shirts this year. I love my t-shirts from GTC they are so cool. That means I had to go and purchase them at the NVIDIA store instead, which I did because they are so cool. It would be great to see a shirt again next year.
I thinks it’s really cool that as the number of speakers continues to grow that they are still nice enough to get speakers a gift. This year it was a nice notebook and pen set.
I am super excited about the NVIDIA Jetson Nano and it was great that the supply of them lasted through the whole show. Purchasing them was super simple, and the badge scanning to keep numbers in check was pretty slick too. I purchased 3 of them. Two at the end of the keynote, and one at the NVIDIA store.
The food this year was fair to midland, or fairly decent as far as conference food goes. I still don’t like having to go down those back steps to the South Hall. I think those steps are a liability waiting to happen, and this is my third year of saying that. I’m glad to the best of my knowledge no one has gotten hurt on those stairs.
One thing I continue to really appreciate that is unique, among the conferences I attend, is the use of drink tickets. It really encourages responsible drinking among attendees and NVIDIA should be commended for that.
Something that would be an awesome thing that is probably a pipe dream, is for people like me who extensively blog the sessions I attend, would be a row of tables up front in conference rooms so I don’t have to balance my device on my leg while listening to the session.
I thought it was great that the facility offered a prayer room and nursing room for those attending GTC. I think that really embraces the diversity of GTC attendees and removes barriers to attending the show. Kudos to both the event center and the show team for making these available and making sure they were publicized.
With that I think it’s about time I wrap up my wrap up post. I got to make a lot of new friends and renew my friendship with so many others. I can’t wait to see everyone next year at GTC20.
May your servers keep running and your data center always be chilled.
Yesterday, March 18th, the NVIDIA Jetson Nano was announced at GTC. As you left the keynote (or when you got back to the San Jose Convention Center) you could purchase one for $99. I purchased a Nano and thought I’d give you a quick look at it.
Below are is a gallery of shots I took of the Jetson Nano and all the stuff that comes in the box which isn’t much.
The unit does not come with a power supply, though it uses a Micro USB connector like most other small compute devices such as Raspberry Pi’s. It also does not have a wireless network built in, and it needs a micro SD card for the OS.
To get started you can visit https://NVIDIA.com/JetsonNano-Start (note this is case sensitive, and if you want to see a cute 404 error get the capitalization wrong).
The Jetson Nano page walks you through setting it up in 6 steps. Which all look very straight forward. The most complicated of which is flashing the SD card.
Some of the things that jumped out at me as I look at the Jetson Nano are that it has both a full sized DisplayPort and an HDMI video output. I’m curious about that value of one over the other.
Another area of interest is that the heat sink has holes drilled for a cooling fan and the base board has a connector that support the addition of a fan. Speaking of the base board, the Jetson Nano is attached to a base board allowing for the NVIDIA chip to be removed (with a screwdriver). The base chip seems to use the DDR4 laptop format for the slot between the base board and the chip, or at least that’s what the connector shows. The micro SD card slot is on the chip its self and not on the base board of the Jetson Nano.
It has a 40 pin expansion header (GPIO, I2C, UART) for different sensors as well as a MIPI-CSI camera connector. There is also a jumper that disables the the USB power input and the allows only the use of the 5V DC power input.
I’d power it up here in my hotel room but I left all my display cables back in Kansas and I dont have a keyboard with me.
Thought I’d share with folks. I haven’t decided what I’m going to do yet with the Jetson Nano but what ever it is it will be fun.
May your servers keep running and your data center always be chilled.
You’ve probably read some blog posts or even case studies on VDI by day and compute by night (cycle harvesting), specifically with GPUs, but you really want to see a real implementation of it or try it for yourself. Well, you’re in luck, you’ve found the right blog!
Over the weekend I finished writing a PowerCLI script to do VDI by day and compute by night with vGPU based VMs. In in this post I’m going to share the script and how to set it up with you.
For those not familiar with the term VDI by day and compute by night here is my example of it. Let’s say you have a bunch of VDI users who are structural engineers. They typically work from 8 in the morning to 5 or so at night on their VDI desktops. They have programs like like AutoCAD which require high end GPUs, but since they are using VDI they’re virtual GPUs (vGPUs). That means you have these high end GPUs that might be sitting idle for 12 to 16 hours a day and the organization spent a lot of money for those GPUs and the rest of the compute resources.
Now what if you could use those GPUs for something other than just engineers virtual desktops? You know, get a little more millage out of them. The R&D department keeps wanting more servers for their high performance computing (HPC) farm… What could 12 extra hours a day of GPU time coming from the Engineers idle VDI hosts do to help them?
That is my VDI by day compute by night or cycle harvesting example. It is re-purposing compute resources from one task (VDI) to supplement another task (HPC, machine learning, deep learning, etc.) when the first task isn’t using them. Further more it’s releasing resources from the second task when they are needed by the primary task.
That’s where the script I wrote comes into play. Trying to do all of the above tasks manually is a pain… Let’s automate it. You are probably thinking, that’s so complex to figure out. And besides if I shutdown my compute VMs after they are half way through a process I’ve lost tons of information and time.
This script addresses these issues and, the release, as of this writing, does it in under 200 lines of code. That said let’s dig into the script.
The script is a PowerCLI script that calls the carrying capacity function I wrote about a few weeks ago. The essence is it performs a loop looking for excess resources of a given vGPU profile (that means the vGPUs used for VDI and Compute can be different), if it finds spare resources the script resumes suspended compute VMs. It continues doing this until the vGPU capacity defined in the script is reached. Then, as VDI desktops start to come back online it starts working in reverse suspending compute VMs from the newest to the oldest (last in, first out (LIFO)). The program loops through balancing of resources for a number of iterations over a given period of time. The net result is an automated instance of VDI by day and compute by night (or whenever there are sufficient free resources, hence why this is commonly called cycle harvesting). A partial flow chart of the logic is shown in the diagram below (there is a lot more happening in the script than whats shown in the diagram).
The order of operations are important in the script. The script first looks at if it needs to suspend compute VMs first, as the primary job of the environment is ‘VDI.’ This insures that resources are allocated to the priority before even considering if it should resume any compute VMs. Then, if there is capacity, compute VMs are started or resumed. And if neither option is possible the environment has reached a “steady state” where resources are being consumed in an optimal manner which prioritizes the ‘VDI’ workload over the compute workload.
This is version 0.1 of the script, so there are a lot of refinements still to come. The script shows that the concept works and delivers the expected results. Future iterations will be more robust and will build on this basic model.
I built and tested this function on VMware PowerCLI 11.0.0 build 10380590 and on PowerShell 5.1.14409.1005. It should be backwards compatible several generations back to the point that the vGPU device backing was added in PowerCLI. Though I’m not sure when that was.
The beginning of the script contains all the variables needed to control it. You can see these in the code snippet below.
#Import vGPU capacity function
. 'C:\Users\Administrator\Desktop\vGPU System Capacity v1_3.ps1'
#define the paramaters
# VDI Side
$SpareVMcapacity = 1 #How many spare VMs should be able to be powered on
# Compute Side
$ComputeVMbaseName = "Compute" #The base name of compute VMs, a three digit number will be added at the end
$ComputeCountFormat = "000" #The preceding zeros in the compute name, so the 6th VM would be Compute006
$MaxComputeVMs = 4 #Total Number of Compute VMs in use
$ComputevGPUtype = "grid_p4-2q" #Which vGPU is in the Compute VM (later I will detect this)
# Opperations Side Varables
$WorkingCluster = "Horizon" #Name of the cluster that should be
$SecondsBetweenScans = 30 #How long will the program wait between scans
$NumberOfScansToPreform = 10 #How many times should the scan be run
###############################
###############################
# Operational variables do not touch
$vGPUslotsOpen = 0 #How many vGPU VMs are currently on
$POComputeVMcount = 0 #Number of powered on (PO) compute VMs
$ScanCount = 0 #How Many times the scan has been run
$CurrVMName = "" #Current VM Name
$ComputeSteadyState = 0
Now lets walk through how to configure the script. To start with we will need the vGPU System Capacity function. You can download that from my GitHub vGPU Capacity repository. Then you want to include the path to that file. (Being sure to mind the dot operator at the beginning of the line.)
Next we define how many spare VMs should be able to be powered on. I currently use one you will probably want to set this as a derivative value of your fail over capacity. Lets say you have N+1 fail over capacity in your environment that means you can have a single host fail and still be functional. To calculate that you would want to multiply the number of vGPUs a profile supports per card times the number of GPUs in a host times the number of spare hosts you have. You may also want to add in some extra capacity as well since it can take a few moments to suspend a compute VM. Example: The P4-2Q profile supports 4 vGPUs per card and I have one card per system with a fail over capacity of N+1, so I would set this value to 4. (4 * 1 * 1 = 4) Thus allowing all VMs from one host to fail to the other host.
The next set of values are for the compute VMs. The first variable is for the base name of the virtual machines used for compute. In the script I call them “Compute.” Which corresponds to the compute VMs base name in my vCenter. This is followed by the trailing digit format for the compute VMs. I’ve defined this as “000.” These two variables are then concatenated in the script to define the compute VMs. Example: Compute + number (formatted in 000 form) = Compute001. This is the name of the second compute VM used.
After we have defined the format of the compute names, we set how many compute VMs there are. I used 4 in this example. It should be noted that the script uses base zero counting so the first VM would be Compute000 and the last would be Compute003. Lastly we define which vGPU the compute VMs will be using. In future iterations of the script I will just pull the vGPU information from the VMs.
Following that section we have the operations variables we need to define. The first one is the name of the cluster we are working in, in my case I call my cluster “Horizon.” Next we define how often we should be scanning for changes in the environment. I went with 30 seconds but your environment may be different and may need sped up or slowed down. Lastly how many times should the scan run. I didn’t build an infinite loop into the system just for stability purposes and because this is an example script, you can modify the code if you want it to run indefinitely.
With those items set the script is ready. Now to prep the environment. The first thing we want to do is setup our compute VMs. That means building out whatever compute nodes you need for your environment connecting them into it’s management and making sure workloads are running correctly on them. You also want to make sure they follow the naming convention we used in the script settings along with the GPU type.
You might be going, but I don’t have the resources to have all these compute VMs powered on at once. That’s fine you can do it in batches. In my case I would power on one Compute VM and configure it. Then I would suspend the VM. By suspending it I’m maintaining its state but releasing its resources. I rinse, lather, and repeat this for all my remaining compute VMs and should wind up with all of the compute VMs configured and in a suspended state.
The thing about suspended VMs is their state is maintained but their resources are released. This means we can setup everything for use and then suspend the VM. Anything in process in them is maintained when suspended. Allowing any operations to complete once resumed.
Now comes a VERY important part. Licensing! Take a look at your VMware Horizon licensing agreement. It probably says something like you can only use Horizon for virtual desktop instances… That means your compute VMs need to be desktops or you need to license your hosts as standard ESXi assets. I can’t tell you which is best to do. You should work with the VMware Licensing team to determine the best course of action. It’s up to you to make sure you fully comply with the terms of your license agreement.
At this point everything should be in place and ready to go. If you so choose you can connect to your vSphere environment and run the example script you configured. When you run it, you will see it determine the state of the environment (suspend compute, resume compute, or balanced) and if there is capacity for it to resume an additional compute VM, it will resume the first compute VM (000). If that balances the resource utilization it will then continue to monitor the environment. Should the minimum capacity be exceed, (because another VDI desktop started up) it will suspend the compute host that was resumed last (LIFO). It will continue balancing the resources in the environment till it reaches the end of its cycles, then it will suspend the compute VMs.
With that you have a running VDI by day and compute by night (or really whenever there are spare resources). Obviously there are many enhancements left to make to this example script, but it validates the basic concepts of cycle harvesting in a virtual environment.
Please leave comments below on what enhancements you would like to see added to this script. And if you would like to contribute, please join me on GitHub.
May your servers keep running and your data center always be chilled.
Update 3-17-19: One of my good friends reminded me one licensing option to consider for your ESXi hosts is the VMware vSphere Scale-Out licensing. You can read more about it here: https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/solutions/vmware-vsphere-scale-out-datasheet.pdf Now depending on your ESXi hosts and the requirements for your VDI environment this licensing bundle may not be what you need. Again it’s best to talk with your VMware rep to figure out what’s best for your environment.
Also after completing this post I created a video demo of my script at work. You can see it below. There are 3 important areas on the screen. Starting at the left, there is the vCenter with 4 compute VMs and 2 holding VMs. In the center is the VDI by day compute by night script. Lastly on the right side is a little script I created to randomly power on and off the holding VMs to emulate a VDI environment with desktops being instantiated and destroyed. The 2 holding VMs don’t contain any OS as the important part of this demonstration is to reserve vGPU resources, after all VDI and VMs are fairly well understood. The Compute VMs are CentOS 7 Linux VMs using Boinc (https://boinc.berkeley.edu/trac/wiki/DesktopGrid) for a workload. The demonstration size is so small because I only have a single host with a single P4 GPU in my lab. You can watch the holding VMs power on and off and the Compute VMs respond to this, though not super fast as the suspend process takes some time. Enjoy.
I’ve been working on a PowerCLI function for the last few months in my free time and now I’d like to share it with everyone. This is a pretty spiffy function for those folks working with vGPUs, and it’s not just for VDI it can help those looking to virtualize ML/DL systems too. (Think VDI by day compute by night.) The function calculates the vGPU carrying capacity of a vSphere environment.
What that means is, if I have an environment with with 30 hosts. Each of which have a bunch of NVIDIA GPUs in them. And on those hosts are a bunch of VMs with various vGPU profiles attached. This function will determine how many more VMs can be powered on with a given vGPU profile.
Yup you read that right, how many more VMs using a given vGPU profile can be powered on.
Now lets make it a bit more interesting. What if you could specify the cluster, or any VI container, in your vSphere environment for that calculation? Well the function I’m sharing does that too.
Let’s take it a step further, what if I only want to calculate my vGPU carrying capacity for powered on hosts, or hosts in maintenance mode, or disconnected hosts? The function will do that to…
How about mixed GPUs? For example having M60s, P40s, and P4s all in the same environment. The function deals with it all.
Feel free to do what ever you want with the script I’ve published it under the GNU version 3 license so its free to do whatever you want to with it.
I’m not going to spend a much time talking about the concepts and constructs of it, but I will spend some time talking about how to use the function.
The bulk of the script file is a function that does all the heavy lifting. The function, vGPUSystemCapacity, takes three arguments. One is required, and the other two are optional. The function returns the number of VMs that can be started with the given profile and if an error were to occur it would return a -1 value.
vPGUSystemCapacity vGPUType as String [vGPULocations as string] [vGPUHostState as string {connected,disconnected,notresponding,maintenance}]
returns int [-1 on error]
The required argument is a string corresponding to the vGPU profile in the format of “grid_p40-2q” The format is “grid_” followed by the physical GPU type “p40” followed by a dash followed by the vGPU profile “2q.” The vGPU profiles can be found in the NVIDIA vGPU User Guide. This is shown in the following example of a function call requesting the results of a “grid_p40-2q” vGPU profile:
vGPUSystemCapacity "grid_p40-2q"
200
Invalid vGPU profiles do not cause errors, so if you were to pass the function a value of “ColdPizza” for a vGPU card type the function will return a 0 value as the system can not support any “ColdPizza” type vGPUs.
vGPUSystemCapacity "ColdPizza"
0
When the function is called with two arguments, the second argument is a string that corosponds the the VIcontainer[] object (ie cluster) you want to calculate the carrying capacity of. For example if I have a cluster named “production” I would pass that to the function as it’s second argument when using the function. You can also pass a wild card character to capture all valid VIcontainers[]. When no argument is passed for the second argument an “*” is the default value. This is to include everything in the vSphere environment. The example below builds on the previous example capturing only vGPUs in the cluster “production.” You can read more about the VIcontainer type on PowerCLI cmdlet reference.
vGPUSystemCapacity "grid_p40-4q" "Production"
100
The third variation of the function takes into account the host state when calculating the carrying capacity. The third value is a VMHostState[] value that is passed to the function as a string. The valid values for host state are “connected”, “disconnected”, “notresponding”, and “maintenance”. You can read about these in the PowerCLI cmdlet reference document as well. The cool thing about these states is you can string them together as a comma delimited list to capture multiple state types all at once. When no string is passed the function defaults to “connected,disconnected,notresponding,maintenance” and will gather all host states. continuing on from our previous example, if we wanted to see the vGPU carrying capacity for connected hosts and hosts in maintenance mode we would use a function like this.
I built and tested this function on VMware PowerCLI 11.0.0 build 10380590 and on PowerShell 5.1.14409.1005. It should be backwards compatible several generations back to the point that the vGPU device backing was added in PowerCLI. Though I’m not sure when that was.
That gets you though working with the vGPUSystemCapacity function I created. If you’ve made it this far you may already have some ideas about what you can do with this. Here are somethings I’d like to do with it.
Use it to monitor how many more VMs of a given type I can instantiate on a system
Capture usage patterns through out the day, letting me know when I am at peek vGPU utilization in my environment
Use this as a core function to enable VDI by day and compute by night.
Let’s touch on this last bullet a bit. VDI by day and compute by night is a term lots of folks are throwing around these days. In fact I’ve done a blog on it myself. The premise is very simple, GPUs are expensive why let them sit idle in the data center when no one is using them? Capture back that time by letting them crunch on some business problems, traditionally at night. To do that we need to know one important thing. At any given time how many vGPUs of a given type are available to perform compute tasks?
Now if only there were some sort of function that could tell me that number… then maybe it would be possible to create a PowerCLI script that could manage all of that for me… Hmmm… I wonder what I’m working on next???
That gets us to the end of this post. Hopefully this script is helpful. If you have improvements post them below or on github. If you run into questions about using the script drop me a note in the comments below. And if you do something cool with it please be sure to share it with the community.
May your servers keep running and your data center always be chilled.
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here:
Cookie Policy
Statements
The fine print: The views expressed herein are solely those of the author and contributing data feeds, these do not reflect or represent the views of employers, associates, associated business entities, groups, organizations, or any other entity (listed or otherwise). Further more some material on this website is dynamically generated and the author does not control its content and is not liable for said content. Content is property of its respective owners. All rights reserved. All material is provided as is, where is, with no expressed or implied warranty of fitness, usability, or merchantability.
In the interest of full disclosure, I am currently a free agent. The views expressed herein do not represent the views of any organization or its subsidiaries.
