The future of high-performance computing will be virtualized, VMware’s Uday Kurkure has told The Register.
Kurkure, the lead engineer for VMware’s performance engineering team, has spent the past five years working on ways to virtualize machine-learning workloads running on accelerators. Earlier this month his team reported “near or better than bare-metal performance” for Bidirectional Encoder Representations from Transformers (BERT) and Mask R-CNN — two popular machine-learning workloads — running on virtualized GPUs (vGPU) connected using Nvidia’s NVLink interconnect.
NVLink enables compute and memory resources to be shared across up to four GPUs over a high-bandwidth mesh fabric operating at 6.25GB/s per lane compared to PCIe 4.0’s 2.5GB/s. The interconnect enabled Kurkure’s team to pool 160GB of GPU memory from the Dell PowerEdge system’s four 40GB Nvidia A100 SXM GPUs.
“As the machine learning models get bigger and bigger, they don’t fit into the graphics memory of a single chip, so you need to use multiple GPUs,” he explained.
Support for NVLink in VMware’s vSphere is a relatively new addition. By toggling NVLink on and off in vSphere between tests, Kurkure was able to determine how large of an impact the interconnect had on performance.
And in what should be a surprise to no one, the large ML workloads ran faster, scaling linearly with additional GPUs, when NVLink was enabled.
Testing showed Mask R-CNN training running 15 percent faster in a twin GPU, NVLink configuration, and 18 percent faster when using all four A100s. The performance delta was even greater in the BERT natural language processing model, where the NVLink-enabled system performed 243 percent faster when running on all four GPUs.
What’s more, Kurkure says the virtualized GPUs were able to achieve the same or better performance compared to running the same workloads on bare metal.
“Now with NVLink being supported in vSphere, customers have the flexibility where they can combine multiple GPUs on the same host using NVLink so they can support bigger models, without a significant communication overhead,” Kurkure said.
HPC, enterprise implications
Based on the results of these tests, Kurkure expects most HPC workloads will be virtualized moving forward. The HPC community is always running into performance bottlenecks that leaves systems underutilized, he added, arguing that virtualization enables users to make much more efficient use of their systems.
Kurkure’s team was able to achieve performance comparable to bare metal while using just a fraction of the dual-socket system’s CPU resources.
“We were only using 16 logical cores out of 128 available,” he said. “You could use that CPU resources for other jobs without affecting your machine-learning intensive graphics modules. This is going to improve your utilization, and bring down the cost of your datacenter.”
Broadcom to buy VMware ‘on Thursday for $60 billion’
By toggling on and off NVLink between GPUs, additional platform flexibility can be achieved by enabling multiple isolated AI/ML workloads to be spread across the GPUs simultaneously.
“One of the key takeaways of this testing was that because of the improved utilization offered by vGPUs connected over a NVLink mesh network, VMware was able to achieve bare-metal-like performance while freeing idle resources for other workloads,” Kurkure said.
VMWare expects these results to improve resource utilization in several applications, including investment banking, pharmaceutical research, 3D CAD, and auto manufacturing. 3D CAD is a particularly high-demand area for HPC virtualization, according to Kurkure, who cited several customers looking to implement machine learning to assist with the design process.
And while it’s possible to run many of these workloads on GPUs in the cloud, he argued that cost and/or intellectual property rules may prevent them from doing so.
vGPU vs MIG
An important note is VMware’s tests were conducted using Nvidia’s vGPU Manager in vSphere as opposed to the hardware-level partitioning offered by multi-instance GPU (MIG) on the A100. MIG essentially allows the A100 to behave like up to seven less-powerful GPUs.
By comparison, vGPUs are defined in the hypervisor and are time-sliced. You can think of this as multitasking where the GPU rapidly cycles through each vGPU workload until they’re completed.
The benefit of vGPUs is users can scale well beyond seven GPU instances at the cost of potential overheads associated with rapid context switching, Kurkure explained. However, at least in his testing, the use of vGPUs didn’t appear to have a negative impact on performance compared to running on bare metal with the GPUs passed through to the VM.
Whether MIG would change this dynamic remains to be seen and is the subject of another ongoing investigation by Kurkure’s team. “It’s not clear when you should be using vGPU and when we should be running in MIG mode,” he said.
More to come
With vGPU with NVLink validated for scale-up workloads, VMware is now exploring options such as how these workloads scale across multiple systems and racks over RDMA over converged Ethernet (RoCE). Here, he says, networking becomes a major consideration.
“The natural extension of this is scale out,” he said. “So, we’ll have a number of hosted connected by RoCE.”
VMware is also investing how virtualized GPUs perform with even larger AI/ML models,
Kurkure’s team is also investigating how these architectures scale with even larger AI/ML, like GPT-3, as well as how they can be applied to telco workloads running at the edge. ®
Swansea City Council has been forced to extend an IT service provider contract to keep its unsupported and unpatched ERP system up and running because its replacement is running two years behind.
A procurement document published last week shows Infosys was awarded £2 million contract (c $2.40 million) extension, until 30 November 2023, to support the Welsh council’s Oracle eBusiness Suite ERP system while it waits for the replacement Oracle Fusion system to be ready. It takes Infosys’s total for supporting the old system to £6.7 million (c $8.1 million).
Council risked failing its Public Service Network accreditation: report
It said using unsupported software “increased the risk of cyber-attacks and potential data theft” while there was also “a risk payroll may not function, staff and pensioners may not be paid.” The report also said the council risked failing its Public Service Network (PSN) accreditation, which meant it could be prevented from sharing data with the health service, police, and the Department for Work and Pensions (DWP).
However, in March 2020, the council invoked a force majeure clause – which alters parties’ contractual obligations – with the support provider Infosys and began discussions to resume the program.
It opted to suspend the program and start back up in February 2021, with the aim to go live in October 2021. The plans said Infosys had agreed to absorb additional costs for this extension.
It said Oracle had agreed to extend support from November 2020 to 2022 so it could get regular updates and patches. “Although this risk still exists, it has been mitigated,” it said.
Given the council has awarded a contract well after the planned go-live of its replacement, it seems those assumptions are under threat. The council has so far failed to respond to The Register‘s request for comment.
Its Fusion project might provide some lessons for the London Borough of Waltham Forest, which plans to have its core solution – a move away from an ageing SAP ERP system – live within a year of the project’s start.
Sony aims to boost the ‘growth of gaming culture’ with two 27-inch monitors and three headsets, designed for both PC and Playstation gamers.
Sony has announced a batch of new monitors and headsets with a focus on PC gamers, as the company looks to reach out to more than its core Playstation audience.
The Inzone range consists of two 27-inch monitors and three headsets, which are all designed to enhance a gamer’s experience. While the main target appears to be PC gamers, the products have features that suit PS5 users.
Sony said its Inzone M9 monitor has 4K resolution and a high contrast with full array local dimming, designed to boost the detail of gaming scenes in deep black and brightness. The monitor also has a 144Hz refresh rate, an IPS display and a 1ms response time. Sony said the monitor will help lead to quicker reactions, which is a clear benefit for competitive PC gamers.
Meanwhile, the M3 monitor will have a 240Hz refresh rate, along with variable refresh rate technology to help gamers “capture movements of rivals in shooter games”.
To go with the monitors, Sony is releasing two wireless headsets, along with the wired Inzone H3 model. The Inzone H9 will have 32 hours of battery life, while the H7 model will have 40 hours.
Speaking on the products, Sony’s head of game business and marketing office Yukihiro Kitajima said there has been a higher interest in gaming with the spread of e-sports tournaments and the advancement of gaming entertainment.
“With Sony’s strong history of high-end audio and visual technology products, we believe this new line will offer even more options for those looking to upgrade their current gaming systems,” Kitajima said.
“We are committed to contributing to the growth of gaming culture by providing PC and PlayStation gamers with a wider range of options to enrich lives through gaming.”
The Inzone headsets range from €300 to €100 and are expected to launch in July, while the M9 monitor is due to launch in the Summer at a cost of €1099. Sony said the pricing and availability of the M3 monitor is expected to be revealed sometime this year.
10 things you need to know direct to your inbox every weekday. Sign up for the Daily Brief, Silicon Republic’s digest of essential sci-tech news.
Many American womenin recent days have deleted period tracking apps from their cellphones, amid fears the data collected by the apps could be used against them in future criminal cases in states where abortion has become illegal.
The trend already started last month when a draft supreme court opinion that suggested the court was set to overturn Roe v Wade was leaked, and has only intensified since the court on Friday revoked the federal right to abortion.
These concerns are not baseless. As with various other apps, cycle trackers collect, retain and at times share some of their users’ data.In a state where abortion is a crime, prosecutors could request information collected bythese apps when building a case against someone. “If they are trying to prosecute a woman for getting an illegal abortion, they can subpoena any app on their device, including period trackers,” said Sara Spector, a Texas-based criminal defense attorney, and ex-prosecutor.
Cycle trackers are popular for a reason. Nearly a third of American women have been using them, according to a 2019 survey published by the Kaiser Family Foundation. They have helped make women’s lives easier in many ways, from family planning and detecting early signs of health issues to choosing the perfect time for a holiday.
A 2019 study published in the British Medical Journal (BMJ) found that 79% of health apps available through the Google Play storethat were related to medicine, including apps that help manage drugs, adherence, medicines, or prescribing information, regularly shared user data and were “far from transparent”. But many of the big players have made progress over the past years.
Two of the most popular period trackers in the US, Flo and Clue, have more than 55 million users combined. The Berlin-based app Clue said it was “committed to protecting” users’ private health data and that it was operating under strict European GDPR laws. The company’s website says the app collects device data, event and usage data, in addition to a user’s IP address, health and sensitive data it may use for the purpose of improving the app, the services, and preventing abusive use of its service. But Clue does not track users’ precise location, and says it does not store sensitive personal data without a user’s explicit permission. The company also tweeted that it would have a “primary legal duty under European law” not to disclose any private health data and it would “not respond to any disclosure request or attempted subpoena of their users’ health data by US authorities”.
But just because data is being processed by a European company, doesn’t mean that it is entirely immune from US prosecution, said Lucie Audibert, a lawyer at Privacy International, a global NGO that researches, litigates and advocates against abuses of technology and data by governments and corporations.
“The fact that GDPR applies is not that relevant in this case. When it comes to a legitimate legal request from US authorities European companies usually comply. Also, a European company may be hosting data outside the EU, making it subject to different legal frameworks and cross-border agreements,” Audibert added. She also stressed that using a Europe-based app won’t protect women from the courts requesting data from them directly. But it can be a slightly better option than using a US-based one because US companies are more easily compelled to comply with American authorities and courts’ requests. Enforcement is more difficult against European ones.
On Friday, Flo announced that it will soon be launching an “Anonymous mode” that can help keep users’ data safe in any circumstances.
The company did not respond to a request for comment.
Stardust did not immediately respond to a request for comment.
Planned Parenthood encourages people to use their app Spot On. “People who want to track their periods and birth control always have the option to remain anonymous by using the Spot On app without creating an account,” the organization said in a statement.“This way, period or birth control data is only saved locally to a person’s phone and can be deleted at any time by deleting the app.”
Third-party apps are not the only option when it comes to period trackers. Apple has a built-in cycle tracker in its Health app that offers more privacy than most external apps. With just a few steps, one can turn off the storing of their health data in iCloud, and it also has the option to store the encrypted data on their computer or phone.
Evan Greer, deputy director of the non-profit advocacy group Fight for the Future, saidthe best way to protect sensitive health data was to only use apps that store data locally rather than in the cloud. “Because any app where a company [that could receive a subpoena] has access to their users’ data could make it vulnerable for a legal request.”
Eva Blum-Dumontet, a tech policy consultant, said, “It is normal that in times of concern, people are looking differently at technology and apps that we trusted.
“I think when there is a discourse around whether women should delete these apps, we have to think about why they use them in the first place,” Blum-Dumontet said. “These trackers help them manage menstrual cycle when they are experiencing pain.”
Blum-Dumontet stressed that instead of asking users to change their behaviors, “it is period trackers that should change their practices”.
“They should never have owned so much data in the first place. If they adopted practices like storing data locally and minimizing the data to what’s strictly necessary we wouldn’t be having this debate now. It’s not too late for them to do the right thing,” she said.
Melissa, a 27-year-old mother from Texas who is goingby only her first name to not jeopardize her employment, said she deleted the app because she fears that when she travels, her state could use her missed period data against her.
“I will miss using the app so much. I have used it for so many things, like tracking my ovulation or predicting my mood changes. Sometimes I wake up feeling irritable, and I don’t know why until my app tells me that this could be normal at this point of my cycle,” she added. Melissa also says she would have loved to use it for future conceptions, but now she can’t.
Although much of the warnings on Friday were focused on just period trackers these are not the only apps that can be used against users when it comes to criminal prosecution,experts warned.
“Google Maps or a random game on your phone could just as easily be weaponized against someone as a menstrual tracking app,” Greer said. “While we need to educate each other and take precautions, it’s not OK to put the responsibility solely on individuals. Companies and lawmakers need to act immediately to protect people.”
The concerns over period tracking data are part of a broader conversation about the amount of personal information smartphones collect. Women’s rights organizations all over the world are warning users to be more mindful of their digital presence, not just when it comes to period trackers.
Cycle tracking apps can be hugely useful for many women, said Jonathan Lord, UK medical director for MSI Reproductive Choices. “But all data can be used against you.”
According to Lord, this danger will remain until “we treat abortion like all other healthcare – regulated like all other medical procedures, but not criminalized”.