Do not pip install gdal, do not install GDAL inside a virtual-env. Instead, use Conda.
These are my instructions on how to install GDAL using Conda on a Mac. Before we dive in, let me explain why I am writing this guide. GDAL stands for the “Geospatial Data Abstraction Library” and it is released by the Open Source Geospatial Foundation. For Python, the GDAL package is released with a package called
osgeo
as well. And as it happens, I need both for a project I’m doing. And I want them neatly wrapped up inside a virtual environment.You can surely use any Python IDE of your preference, but the best Python IDE will facilitate our work a lot because the suggested configuration will help you avoid mistakes and save time on typing, running, and debugging your code. If you need a full-fledged and cross-platform Python IDE, then you should stick to PyCharm, WingIDE, or PyDev. Feb 27, 2020 PyCharm Community Edition, Vim, and Visual Studio Code are probably your best bets out of the 22 options considered. 'Sophisticated autocompletion ' is the primary reason people pick PyCharm Community Edition over the competition. This page is powered by a knowledgeable community that helps you make an informed decision.
I started by trying to use pip and virtual-env. This obscure how-to (and first post on Medium!) comes to you after hours of googling and trying to debug the errors I encountered along the way. In the end, I shifted gears, tried something new and switched over to Conda. This was the solution to all my problems. OK, except a few, but I fixed those too.
Conda is great because it’s a package manager like pip, but it also manages your virtual environments like virtual-env does. Except it does both way better and it’s a joy to use. Treat yo self and switch to Conda.
I wanted to spare you the trouble I experienced so I wrote up the following instructions.
Environment:
A quick note about the environment I’m working in:
Mac OS 10.12.6 Sierra
Miniconda2 for Python 2 (Conda 4.3.30) info
Python 2.7.14
optional python packages:
Jupyter notebook, installs with$ conda install jupyter
nb_conda, makes Jupyter play nice with Conda,$ conda install nb_conda
If you are working in a different development environment, your mileage may vary.
And now, the instructions that I myself needed hours ago…
How to Install GDAL ? using Conda ? on a Mac ?
This tutorial assumes you have Conda already installed and a Conda environment already created. Instructions here and here, respectively.
Step 1: Activate your Conda environment ?
Open up Terminal, run this:
For me, [yourEnvironmentName] =
geoenv
.(We can deactivate with the command
$ source deactivate
.)Step 2: Ok, now we get to install GDAL. ?
Still in Terminal, run this command:
Here’s what I got:
Great. As it turns out, for the osgeo subpackage to work, we actually need the dependency
jpeg version 8
, rather than 9
. You can read more about how I came to that conclusion towards the end of this post, under #Diagnosing.For now, all you need to do is run this:
The “-f” flag forces the install (which is really a downgrade of the the jpeg module).
OK, we should have a working version of GDAL now! Let’s just test it to make sure.
Step 3: Test the installation ?
You can do this in the command line or in a Jupyter notebook. Since I want to make sure gdal will work in Jupyter later, I’m going to test there.
To open a new Jupyter Notebook ?, go back to Terminal, run this command:
This command will open up a new tab in your internet browser with the Jupyter Notebook file viewer. Navigate to the directory where you wish to save your notebook. Now, we want to start a new notebook. Go to the upper righthand corner, click “New”.
Make sure to choose the Conda environment you’ve been working with as the Python Kernel.
Let’s go ahead and test! Run these commands in the notebook.
The help for gdal works, so we’re off to the races.
Success! ?
We’ve finally got GDAL installed as well as osgeo. Everything is working (for now). This somewhat lengthy post was a joy to write, as this problem caused me innumerable hours of strife. I hope to save you from the same. Thanks in advance for your claps ?? Let me know if something needs an edit or clarification. With that, I’m off to explore graphs with
networkx
!With ❤︎,
Filip
p.s.
#Motivation for this post
I generally like to use virtual environments on projects to keep things organized. First, I tried to install GDAL inside a python virtual-env which was a huge fail. There are instructions out there how to do that for Windows and Ubuntu, but I couldn’t get it to work for Mac. Virtual-env was more like virtual-enemy. Some folks on StackOverflow suggested to use Conda instead. I ran into a few snags anyways, so I decided to publish these instructions how to Install GDAL using Python/Conda on Mac. Dear reader, I hope this guide saves you some time.
#What am I using GDAL for
I need GDAL for a very particular reason. It’s required for and a dependency of the
read_shp()
function of the networkx Python module. That function reads in an ESRI shapefile (geospatial data) an converts it into a network/graph object. Obviously, you might need GDAL for something else.To install, I tried using
pip install gdal
inside a Python virtual environment (a.k.a. virtual-env) at first. That failed. I guess you could say it was only a pip dream, sigh. Or maybe it had something to do with having QGIS via Kyngchaos installed. That distribution includes GDAL not as a Python package, but as a Framework.Anyways, the bottom line is that I still needed GDAL to work inside a Python virtual environment.
#Diagnosing the Conda install issue:
It was not possible for me to get GDAL installed inside a virtual-env using pip. That’s why I switched to Conda.
When running the install in Conda, I ran into a few issues. Simply running the
read_shp
function from networkx
was giving me a generic error, much like it was in virtual-env.ImportError: read_shp requires OGR:
In the screenshot you can see that the code requires
from osgeo import ogr
which is actually included as part of the GDAL module.So when we try to
import gdal
, we can see what’s actually happening:Library not loaded: @rpath/libjpeg.8.dylib
The jpeg8 library is not loading. To investigate, we can check what packages conda has installed:
$ conda list
Moreover, when I uninstall and reinstall only
gdal
, it actually becomes evident that gdal itself updates jpeg to version 9, only to break later.The fix is to simply downgrade jpeg 9 to jpeg 8 after installing gdal. You can find the recipe for that in Step 3 of the #Instructions above. Thanks!
Sources I used:
- egayer’s comment in this thread on the gdal GitHub
- I’m not including my crazy, exhaustive searches for anything related to “pip install GDAL” or “GDAL python install mac virtual-env” in this list. Bless your heart if you try to go that path.
Python code editors are designed for the developers to code and debug program easily. Using these Python IDE(Integrated Development Environment), you can manage a large codebase and achieve quick deployment.
Developers can use these editors to create desktop or web application. The IDEs can also be used by DevOps engineers for continuous Integration.
Following is a handpicked list of Top Python Code
Editors, with popular features and latest download links. The list contains both open-source(free) and premium tools.
1) PyCharm
PayCharm is a cross-platform IDE used for Python programming. This editor can be used on Windows, macOS, and Linux. This software contains API that can be used by the developers to write their own Python plugins so that they can extend the basic functionalities.
Price: Free
Features:
- It is an intelligent Python code editor supports for CoffeeScript, JavaScript, CSS, and TypeScript.
- Provides smart search to jump to any file, symbol, or class.
- Smart Code Navigation
- It offers quick and safe refactoring of code.
- It allows you to access PostgreSQL, Oracle, MySQL, SQL Server, and many other databases from the IDE.
Download Link: https://www.jetbrains.com/pycharm/
2) Spyder
Spyder is a scientific integrated development environment written in Python. This software is designed for and by scientists who can integrate with Matplotlib, SciPy, NumPy, Pandas, Cython, IPython, SymPy, and other open-source software. Spyder is available through Anaconda (open-source distribution system) distribution on Windows, macOS, and Linux.
Price: Free
Features:
- Allows you to run Python code by cell, line, or file.
- Plot a histogram or time-series, make changes in dateframe or numpy array.
- It offers automatic code completion and horizontal/vertical splitting.
- Find and eliminate bottlenecks
- An interactive way to trace each step of Python code execution.
Download Link:https://www.spyder-ide.org/
3) IDLE
IDLE (Integrated Development and Learning Environment) is a default editor that comes with Python. This software helps a beginner to learn Python easily. IDLE software package is optional for many Linux distributions. The tool can be used on Windows, macOS, and Unix.
Price: free
Features:
- Search multiple files
- It has an interactive interpreter with colorizing of input, output, and error messages.
- Supports smart indent, undo, call tips, and auto-completion.
- Enable you to search and replace within any window.
Download Link:https://docs.python.org/3/library/idle.html
4) Sublime Text 3
Sublime Text 3 is a code editor which supports many languages including Python. It has basic built-in support for Python. Customization of Sublime Text 3 is available for creating create a full-fledged Python programming environment. The editor supports OS X, Windows, and Linux operating systems.
Price: Free Trial
Features:
- Allows you to highlight syntax.
- It has command Palette implementation that accepts text input from users.
- Handle UTF8 BOMs in .gitignore files
- Display badges for folders and file to indicate Git status
- Changes to a file are represented by markers available in the gutter.
Download Link: https://www.sublimetext.com/3
5) Visual Studio Code
Visual Studio Code (VS Code) is an open-source environment developed by Microsoft. This IDE can be used for Python development. Visual Studio Code is based on Electron which is a framework to deploy Node JS applications for the computer running on the Blink browser engine.
Price: Free
Features:
- The editor provides smart code completion based on function definition, imported modules, as well as variable types.
- You can work with Git as well as other SCM providers
- Enable you to debug code from the editor.
- Provides extensions to add new languages, debuggers, themes to gain the advantage of additional services.
Download Link: https://code.visualstudio.com/
6) Atom
Atom is a useful code editor tool preferred by programmers due to its simple interface compared to the other editors. Atom users can submit packages and them for the software.
Price: Free
Features:
- Package Manager Integrated for Plugins support
- The feature of smart autocompletion
- Supports Command Palette
- Multiple panes
- Allow cross-platform editing
Download Link:https://atom.io/
7) Jupyter
Jupyter is a tool for people who have just started with data science. It is easy to use, interactive data science IDE across many programming languages that just not work as an editor, but also as an educational tool or presentation.
Price: Free
Features:
- It supports for Numerical simulation, data cleaning machine learning data visualization, and statistical modeling.
- Combine code, text, and images.
- Support for many programming languages.
- Integrated data science libraries (matplotlib, NumPy, Pandas).
Download Link:https://jupyter.org/install.html
8) Pydev
PyDev is a third-party Python editor for Eclipse. This editor can be used in not only Python but IronPython and Jython development.
Price: Free
Features:
- It has interactive console shortcuts
- Allows you to create a Google App Engine (GAE) Python project
- Find and Go to definition
- Automatically import code to complete it.
- You can Configure Django integration.
Download Link:https://www.pydev.org/
9) Thonny
Thonny is an IDE for learning and teaching programming, specially designed with the beginner Pythonista scripting environment. It is developed at The University of Tartu, which you can download for free on the Bitbucket repository for Windows, Linux, and Mac.
Price: Free
Features:
- Allows developers to view how their code and shell commands affect Python variables.
- It has a simple debugger.
- Provides support for evaluating an expression.
- Python function call opens a new window with separate local variables table as well as code pointer.
- Automatically spot syntax error.
Download Link:https://thonny.org/
10) Wing
Wing is a lightweight Python environment which is designed to give you productive development experience.
Price: Wing Pro trial is free. Wind Personal and Wing 101 are paid versions.
Features:
- Immediate feedback by writing your Python code.
- Helps you to remove common errors and write better Python code.
- You can check for debug data and try out bug fixes interactively without restarting your app.
- Wing supports test-driven development with various frameworks like the unittest, pytest, nose, doctest, and Django testing.
Download Link:https://wingware.com/
11) ActivePython
Increase software development data science with a secure and supported Python distribution. ActivePython is software consisting of the Python implementation CPython and a set of various extensions to facilitate installation.
Price: Free for community, however, coder, team, business. Enterprise versions are paid.
Features:
- It allows you to connect to your big data and databases, including Redis, MySQL, Hadoop, and MongoDB.
- Helps you to manage your data using, SciPy, Pandas, NumPy, and MatPlotLib.
- Supports machine learning models like TensorFlow, Keras, and Theano.
- Compatible with open-source Python so that you can avoid vendor lock-in.
- Uses OpenSSL patch for security.
Download Link:https://www.activestate.com/products/activepython/