Category: Xcode
Setting up gcc / OpenMP on OSX (Homebrew edition)
Note: This is part of a series of “how-to” blog posts to help new users and developers of BioFVM and PhysiCell. This guide is for OSX users. Windows users should use this guide instead. A Linux guide is expected soon.
These instructions should get you up and running with a minimal environment for compiling 64-bit C++ projects with OpenMP (e.g., BioFVM and PhysiCell) using gcc. These instructions were tested with OSX 10.11 (El Capitan) and 10.12 (Sierra), but they should work on any reasonably recent version of OSX.
In the end result, you’ll have a compiler and key makefile capabilities. The entire toolchain is free and open source.
Of course, you can use other compilers and more sophisticated integrated desktop environments, but these instructions will get you a good baseline system with support for 64-bit binaries and OpenMP parallelization.
Note 1: OSX / Xcode appears to have gcc out of the box (you can type “gcc” in a Terminal window), but this really just maps back onto Apple’s build of clang. Alas, this will not support OpenMP for parallelization.
Note 2: In this post, we showed how to set up gcc using the popular MacPorts package manager. Because MacPorts builds gcc (and all its dependencies!) from source, it takes a very, very long time. On my 2012 Macbook Air, this step took 16 hours. This tutorial uses Homebrew to dramatically speed up the process!
Note 3: This is an update over the previous version. It incorporates new information that Xcode command line tools can be installed without the full 4.41 GB download / installation of Xcode. Many thanks to Walter de Back and Tim at the Homebrew project for their help!
What you’ll need:
- XCode Command Line Tools: These command line tools are needed for Homebrew and related package managers. Installation instructions are now very simple and included below. As of January 18, 2016, this will install Version 2343.
- Homebrew: This is a package manager for OSX, which will let you easily download and install many linux utilities without building them from source. You’ll particularly need it for getting gcc. Installation is a simple command-line script, as detailed below. As of August 2, 2017, this will download Version 1.3.0.
- gcc (from Homebrew): This will be an up-to-date 64-bit version of gcc, with support for OpenMP. As of August 2, 2017, this will download Version 7.1.0.
Main steps:
1) Install the XCode Command Line Tools
Open a terminal window (Open Launchpad, then “Other”, then “Terminal”), and run:
user$ xcode-select --install
A window should pop up asking you to either get Xcode or install. Choose the “install” option to avoid the huge 4+ GB Xcode download. It should only take a few minutes to complete.
2) Install Homebrew
Open a terminal window (Open Launchpad, then “Other”, then “Terminal”), and run:
user$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Let the script run, and answer “y” whenever asked. This will not take very long.
3) Get, install, and prepare gcc
Open a terminal window (see above), and search for gcc, version 7.x or above
user$ brew search gcc
You should see a list of packages, including gcc7. (In 2015, this looked like “gcc5”. In 2017, this looks like “gcc@7”.)
Then, download and install gcc:
user$ brew install gcc
This will download whatever dependencies are needed, generally already pre-compiled. The whole process should only take five or ten minutes.
Lastly, you need to get the exact name of your compiler. In your terminal window, type g++, and then hit tab twice to see a list. On my system, I see this:
Pauls-MBA:~ pmacklin$ g++ g++ g++-7 g++-mp-7
Look for the version of g++ without an “mp” (from MacPorts) in its name. In my case, it’s g++-7. Double-check that you have the right one by checking its version. It should look something like this:
Pauls-MBA:~ pmacklin$ g++-7 --version g++-7 (Homebrew GCC 7.1.0) 7.1.0 Copyright (C) 2017 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Notice that Homebrew shows up in the information. The correct compiler is g++-7.
PhysiCell Version 1.2.2 and greater use a system variable to record your compiler version, so that you don’t need to modify the CC line in PhysiCell Makefiles. Set the PHYSICELL_CPP variable to record the compiler you just found above. For example, on the bash shell:
export PHYSICELL_CPP=g++-7 echo export PHYSICELL_CPP=g++-7 >> ~/.bash_profile
One last thing: If you don’t update your paths, make will may fail as it continues to combine Apple’s “gcc” toolchain with real gcc. (This seems to happen most often if you installed an older gcc like gcc5 with MacPorts earlier.) You may see errors like this:
user$ make g++-7 -march=core2 -O3 -fomit-frame-pointer -fopenmp -std=c++11 -c BioFVM_vector.cpp FATAL:/opt/local/bin/../libexec/as/x86_64/as: I don't understand 'm' flag! make: *** [BIOFVM_vector.o] Error 1
To avoid this, run:
echo export PATH=/usr/local/bin:$PATH >> ~/.bash_profile
Note that you’ll need to open a new Terminal window for this fix to apply.
4) Test your setup
I wrote a sample C++ program that tests OpenMP parallelization (32 threads). If you can compile and run it, it means that everything (including make) is working! :-)
Make a new directory, and enter it
Open Terminal (see above). You should be in your user profile’s root directory. Make a new subdirectory called GCC_test, and enter it.
mkdir GCC_test cd GCC_test
Grab a sample parallelized program:
Download a Makefile and C++ source file, and save them to the GCC_test directory. Here are the links:
- Makefile: [click here]
- C++ source: [click here]
Note: The Makefiles in PhysiCell (versions > 1.2.1) can use an environment variable to specify an OpenMP-capable g++ compiler. If you have not yet done so, you should go ahead and set that now, e.g., for the bash shell:
export PHYSICELL_CPP=g++-7 echo export PHYSICELL_CPP=g++-7 >> ~/.bash_profile
Compile and run the test:
Go back to your (still open) command prompt. Compile and run the program:
make ./my_test
The output should look something like this:
Allocating 4096 MB of memory ... Done! Entering main loop ... Done!
Note 1: If the make command gives errors like “**** missing separator”, then you need to replace the white space (e.g., one or more spaces) at the start of the “$(COMPILE_COMMAND)” and “rm -f” lines with a single tab character.
Note 2: If the compiler gives an error like “fatal error: ‘omp.h’ not found”, you probably used Apple’s build of clang, which does not include OpenMP support. You’ll need to make sure that you set the environment variable PHYSICELL_CPP as above (for PhysiCell 1.2.2 or later), or specify your compiler on the CC line of your makefile (for PhysiCell 1.2.1 or earlier).
Now, let’s verify that the code is using OpenMP.
Open another Terminal window. While the code is running, run top. Take a look at the performance, particularly CPU usage. While your program is running, you should see CPU usage fairly close to ‘100% user’. (This is a good indication that your code is running the OpenMP parallelization as expected.)
What’s next?
Download a copy of PhysiCell and try out the included examples! Visit BioFVM at MathCancer.org.
- PhysiCell links:
- PhysiCell Method Paper at bioRxiv: https://doi.org/10.1101/088773
- PhysiCell on MathCancer: http://PhysiCell.MathCancer.org
- PhysiCell on SourceForge: http://PhysiCell.sf.net
- PhysiCell on github: http://github.com/MathCancer/PhysiCell
- PhysiCell tutorials: [click here]
- BioFVM links:
- BioFVM announcement on this blog: [click here]
- BioFVM on MathCancer.org: http://BioFVM.MathCancer.org
- BioFVM on SourceForge: http://BioFVM.sf.net
- BioFVM Method Paper in BioInformatics: http://dx.doi.org/10.1093/bioinformatics/btv730
- BioFVM tutorials: [click here]
Setting up gcc / OpenMP on OSX (Homebrew edition) (outdated)
Note 1: This is the part of a series of “how-to” blog posts to help new users and developers of BioFVM and PhysiCell. This guide is for OSX users. Windows users should use this guide instead. A Linux guide is expected soon.
Note 2: This tutorial is outdated. Please see this updated version.
These instructions should get you up and running with a minimal environment for compiling 64-bit C++ projects with OpenMP (e.g., BioFVM and PhysiCell) using gcc. These instructions were tested with OSX 10.11 (El Capitan), but they should work on any reasonably recent version of OSX.
In the end result, you’ll have a compiler and key makefile capabilities. The entire toolchain is free and open source.
Of course, you can use other compilers and more sophisticated integrated desktop environments, but these instructions will get you a good baseline system with support for 64-bit binaries and OpenMP parallelization.
Note 3: OSX / Xcode appears to have gcc out of the box (you can type “gcc” in a Terminal window), but this really just maps back onto Apple’s build of clang. Alas, this will not support OpenMP for parallelization.
Note 4: Yesterday in this post, we showed how to set up gcc using the popular MacPorts package manager. Because MacPorts builds gcc (and all its dependencies!) from source, it takes a very, very long time. On my 2012 Macbook Air, this step took 16 hours. This tutorial uses Homebrew to dramatically speed up the process!
What you’ll need:
- XCode: This includes command line development tools. Evidently, it is required for both Homebrew and its competitors (e.g., MacPorts). Download the latest version in the App Store. (Search for xcode.) As of January 15, 2016, the App Store will install Version 7.2. Please note that this is a 4.41 GB download!
- Homebrew: This is a package manager for OSX, which will let you easily download and install many linux utilities without building them from source. You’ll particularly need it for getting gcc. Installation is a simple command-line script, as detailed below. As of January 17, 2016, this will download Version 0.9.5.
- gcc5 (from Homebrew): This will be an up-to-date 64-bit version of gcc, with support for OpenMP. As of January 17, 2016, this will download Version 5.2.0.
Main steps:
1) Download, install, and prepare XCode
As mentioned above, open the App Store, search for Xcode, and start the download / install. Go ahead and grab a coffee while it’s downloading and installing 4+ GB. Once it has installed, open Xcode, agree to the license, and let it install whatever components it needs.
Now, you need to get the command line tools. Go to the Xcode menu, select “Open Developer Tool”, and choose “More Developer Tools …”. This will open up a site in Safari and prompt you to log in.
Sign on with your AppleID, agree to yet more licensing terms, and then search for “command line tools” for your version of Xcode and OSX. (In my case, this is OSX 10.11 with Xcode 7.2) Click the + next to the correct version, and then the link for the dmg file. (Command_Line_Tools_OS_X_10.11_for_Xcode_7.2.dmg).
Double-click the dmg file. Double-click pkg file it contains. Click “continue”, “OK”, and “agree” as much as it takes to install. Once done, go ahead and exit the installer and close the dmg file.
2) Install Homebrew
Open a terminal window (Open Launchpad, then “Other”, then “Terminal”), and run:
> ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Let the script run, and answer “y” whenever asked. This will not take very long.
3) Get, install, and prepare gcc
Open a terminal window (see above), and search for gcc, version 5.x or above
> brew search gcc5
You should see a list of packages, including gcc5. Take a note of what is found. (In my case, it found homebrew/versions/gcc5.)
Then, download and install gcc5:
> brew install homebrew/versions/gcc5
This will download whatever dependencies are needed, generally already pre-compiled. The whole process should only take five or ten minutes.
Lastly, you need to get the exact name of your compiler. In your terminal window, type g++, and then hit tab twice to see a list. On my system, I see this:
Pauls-MBA:~ pmacklin$ g++ g++ g++-5 g++-mp-5
Look for the version of g++ without an “mp” (for MacPorts) in its name. In my case, it’s g++-5. Double-check that you have the right one by checking its version. It should look something like this:
Pauls-MBA:~ pmacklin$ g++-5 --version g++-5 (Homebrew gcc5 5.2.0) 5.2.0 Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Notice that Homebrew shows up in the information. The correct compiler is g++-5.
5) Test the compiler
Write a basic parallelized program:
Open Terminal (see above). You should be in your user profile’s root directory. Make a new subdirectory, enter it, and create a new file:
> mkdir omp_test > cd omp_test > nano omp_test.cpp
Then, write your basic OpenMP test:
#include <iostream>
#include <cmath>
#include <vector>
#include <omp.h>
int main( int argc, char* argv[] )
{
omp_set_num_threads( 8 );
double pi = acos( -1.0 );
std::cout << "Allocating memory ..." << std::endl;
std::vector<double> my_vector( 128000000, 0.0 );
std::cout << "Done!" << std::endl << std::endl;
std::cout << "Entering main loop ... " << std::endl;
#pragma omp parallel for
for( int i=0; i < my_vector.size(); i++ )
{
my_vector[i] = exp( -sin( i*i + pi*log(i+1) ) );
}
std::cout << "Done!" << std::endl;
return 0;
}
Save the file (as omp_test.cpp). (In nano, use [Control]+X, Y, and then confirm the choice of filename.)
In the omp_set_num_threads() line above, replace 8 with the maximum number of virtual processors on your CPU. (For a quad-core CPU with hyperthreading, this number is 8. On a hex-core CPU without hyperthreading, this number is 6.) If in doubt, leave it alone for now.
Write a makefile:
Next, create a Makefile to start editing:
> nano Makefile
Add the following contents:
CC := g++-5 # replace this with your correct compiler as identified above ARCH := core2 # Replace this with your CPU architecture. # core2 is pretty safe for most modern machines. CFLAGS := -march=$(ARCH) -O3 -fopenmp -m64 -std=c++11 COMPILE_COMMAND := $(CC) $(CFLAGS) OUTPUT := my_test all: omp_test.cpp $(COMPILE_COMMAND) -o $(OUTPUT) omp_test.cpp clean: rm -f *.o $(OUTPUT).*
Go ahead and save this (as Makefile). ([Control]-X, Y, confirm the filename.)
Compile and run the test:
Go back to your (still open) command prompt. Compile and run the program:
> make > ./my_test
The output should look something like this:
Allocating memory ... Done! Entering main loop ... Done!
Note 1: If the make command gives errors like “**** missing separator”, then you need to replace the white space (e.g., one or more spaces) at the start of the “$(COMPILE_COMMAND)” and “rm -f” lines with a single tab character.
Note 2: If the compiler gives an error like “fatal error: ‘omp.h’ not found”, you probably used Apple’s build of clang, which does not include OpenMP support. You’ll need to make sure that you specify your compiler on the CC line of your makefile.
Now, let’s verify that the code is using OpenMP.
Open another Terminal window. While the code is running, run top. Take a look at the performance, particularly CPU usage. While your program is running, you should see CPU usage fairly close to ‘100% user’. (This is a good indication that your code is running the OpenMP parallelization as expected.)
What’s next?
Download a copy of BioFVM and try out the included examples! Visit BioFVM at MathCancer.org.
- BioFVM announcement on this blog: [click here]
- BioFVM on MathCancer.org: http://BioFVM.MathCancer.org
- BioFVM on SourceForge: http://BioFVM.sf.net
- BioFVM Method Paper in BioInformatics: http://dx.doi.org/10.1093/bioinformatics/btv730
- BioFVM tutorials: [click here]