How Small AI Companies Can Train Models Sustainably — Every Little Bit Helps!
In today’s world, the push for energy efficiency and sustainability is louder than ever. Whether you’re an eco-warrior or someone who just likes their energy bill to stay under control, this is a topic that’s hard to ignore. But when you dive into the deep end of the AI industry, particularly training those generative AI models we all know and love (looking at you, ChatGPT!), the sheer amount of energy required can make your eco-conscious heart skip a beat.
Training AI models isn’t just about the cool results — it’s about powering machines that guzzle electricity like a thirsty camel at an oasis. In fact, according to the BBC, the energy consumption of training large AI models has a pretty hefty carbon footprint. So how do small AI companies like Aeris UK fit into this picture? What steps are we taking to ensure we’re not just contributing to the problem, but also exploring ways to reduce our impact?
Well, as Tesco likes to say, “Every Little Helps” — and we at Aeris take that to heart. Let’s talk about how we’re working to make AI training a little more energy-efficient and a lot more sustainable.
Enter the DARK HERO: Aeris’s Powerhouse
At Aeris, we’ve taken the leap and set up our very own AI training hardware. We call it DARK HERO (because who doesn’t want to name their server something epic?), and it’s a compact but powerful Ubuntu workstation that handles all of our AI model training. No cloud services for us — this beauty lives right here in the office.
Why? Well, for one, it’s more cost-efficient for small and medium-sized companies. Cloud computing is fantastic when you’re a mega-corporation with deep pockets, but for smaller teams, having a dedicated machine in-house can be a lot friendlier on the budget. Plus, it gives us more control over our data security — a huge deal when you’re dealing with sensitive information.
But here’s the kicker: DARK HERO is on 24/7. That’s right, our machine hums along day and night, just waiting for someone to hop on and use it. And here’s where things get tricky. Like many workspaces, we initially had the standard setup where team members used remote login software (AnyDesk, TeamViewer, etc.) to share access. This meant a lot of juggling: people had to organise who used the machine when, and there were plenty of moments where valuable compute resources were sitting idle, sipping on energy without actually doing anything productive. Talk about a waste!
Better Access, Less Idle Time
One of the first things we did to cut down on wasted energy was to ditch the one-login-for-all method. We set up SSH access instead, so everyone on the team could log in with their own credentials. This instantly solved two problems: it allowed for better compliance with cyber security standards (hello, Cyber Essentials!) and meant we could use the machine in parallel, instead of one person monopolising it. The trade-off? We had to pay for a static public IP, but hey, that’s a small price to pay for efficiency!
But, surprise, most teams stop here. And they miss out on another energy-saving trick.
Streamline Your Dev Environment
Beyond shared access, there’s another subtle but impactful change: consolidating library dependencies. Every time a new person joins the team or we switch to a different model, setting up the environment can be a huge time sink. And it’s not just a time issue — every bit of setup, downloading and installing can add to your energy consumption.
To counter this, we took a page from the software development playbook and pre-installed the major packages and dependencies our team uses. Things like CUDA (for those lovely GPUs) and pyenv (for managing Python environments) are already set up and ready to go. This saves everyone time, energy (literally!) and helps ensure the team stays on the same page. Of course, it’s not the simplest thing in the world to set up. It requires organising user groups with different privilege levels, but trust us, it’s worth the effort.
Fighting Idle Time — The Wake-on-LAN Trick
Despite all these changes, we still noticed something: idle energy consumption. Even with SSH access, DARK HERO was often sitting idle, powered on and waiting for the next training session. To monitor this, we used an energy monitor plug (yes, those exist, and they’re a bit of a game-changer). The graph it generated told us what we suspected: energy spikes during training and drops back to a frustratingly flat line when idle.
So, what do you do when your machine’s just wasting electricity between use? Turning it off and on manually isn’t a great solution — it’s inconvenient and people need access at unpredictable times. That’s where Wake-on-LAN (WoL) comes in.
What Is Wake-on-LAN?
Imagine you’re lounging on the couch, and someone walks in, turns on the TV and hands you the remote. That’s Wake-on-LAN. It’s a feature that allows a computer to be powered on remotely by sending a “magic packet” over the network. This means that DARK HERO can be off most of the time, consuming no power, but it’ll spring to life whenever someone needs it.
Benefits of Wake-on-LAN
The beauty of Wake-on-LAN is twofold:
Energy Savings: You no longer have to keep your machines running 24/7. They’re only on when someone’s actively using them, and the rest of the time they’re happily powered down, saving energy and cutting your carbon footprint.
Convenience: You can set it up so that team members can wake the machine up whenever they need it, regardless of time zone or work hours. No one needs to be physically there to hit the power button.
How to Set Up Wake-on-LAN
Setting up Wake-on-LAN is easier than you might think. Here’s a quick rundown of how to get it going:
Enable Wake-on-LAN in BIOS: Start by entering your computer’s BIOS (hit that F2 or Delete key when booting up). Look for the Wake-on-LAN option and make sure it’s enabled.
Configure Network Settings: Your machine’s network card needs to be set up to respond to the magic packet. On Ubuntu, for instance, you can use the ethtool command to verify and enable WoL support.
Set Up WoL Client: Finally, you need a tool to send the magic packet. There are plenty of options available, from apps to command-line tools. Choose one that works for your team, and voilà! You can wake your machine remotely from anywhere.
Conclusion: Every Bit Counts
At Aeris, we’ve learned that even small changes can lead to meaningful reductions in energy usage. From optimising how our team accesses DARK HERO to setting up Wake-on-LAN, we’re doing our part to make AI training a bit more sustainable. It’s not perfect, but as Tesco’s slogan reminds us, “Every Little Helps.”
And in a world where energy efficiency is king, those little changes add up.
Comments