When MemoryError in Python happens, your script will also refuse to proceed. This tutorial will explain this error and suggest common solutions so your Python program can be executed successfully.

What Is MemoryError?

MemoryError is a built-in exception of Python. The interpreter raises this exception when it can’t allocate any more memory to an operation.

There are many scenarios in which you may see this exception. Perhaps your code uses too many objects. The dataset you load may be too big. The computer systems of many people also have a memory capacity too small for serious projects, while others simply use the wrong Python version.

How To Avoid MemoryError In Python

Install The 64-Bit Python Version

The memory limit of 32-bit applications depends on the operating system, platform, and related settings. Windows, for example, has a 2-GB limit for those programs when the IMAGE_FILE_LARGE_ADDRESS_AWARE flag isn’t set.

Even when you have cleared every limitation, your system still has no way to provide your 32-bit Python installation with more than 4 GB of RAM.

On Python3, you can check with the code below:

It will print 32 if you are running a 32-bit Python installation. If this is the case, the obvious solution is to get 64-bit Python. Remember that this requires a 64-bit operating system.

This upgrade will avoid MemoryError in Python by massively expanding the memory address space your Python program can request. On Windows 8.1 or later, this limit will be raised to 128 TB, virtually unlimited for any project.

Get More RAM

Big Python projects demand plenty of resources to run, including memory. An easy way to supply them with more RAM is to upgrade your computer system’s memory capacity.

Buy new RAM sticks if you still have empty slots, or replace them with bigger capacities if you run out of space. Consider buying a whole new computer when your current system is too old and weak. Renting a virtual machine with strong specifications is also a great idea.

Optimize Your Code

When you have poorly-written code without resource optimization in mind, your system may run out of memory no matter how powerful it is. The program gets worse for data scientists or machine learning engineers, who regularly process huge datasets.

Here are some tips on reducing memory usage of your Python program:

• Choose the right data type: objects in Python can use a lot of memory. Don’t abuse dictionaries, for instance, as they can create serious overhead.
• Use techniques like generators: Generator functions in Python create lazy iterators. They allow you to loop over but don’t actually contain their contents in memory. Chunking, compressing, and indexing your data can help prevent MemoryError in Python as well.
• Use specialized libraries for data crunching: Pandas, NumPy, and similar libraries can manage memory better. They can even help you process datasets bigger than your available RAM.

Here is an example: a list of one million zeros and a similar 16-bit array in NumPy. You can see the difference in memory usage between them, measured with the Memory Profiler library.

List:

Output:

NumPy array:

Output:

Conclusion

You will see MemoryError in Python when your system runs out of money. This is a common error when handling large files and datasets. Besides maximizing your system’s specifications, try to optimize your code with special techniques and specialized libraries.

→ Read more: How to Clear Memory in Python?