CPAN on OS X

2024-03-26

CPAN, the Comprehensive Perl Archive Network is a free community provided library of Perl programs. My program expandfile, which I use to generate web pages, uses CPAN modules.

Installing CPAN

Here are instructions for upgrading 64-bit CPAN, tested
April 2014 on 10.9.2 Mavericks,
July 2015 on 10.10.4 Yosemite,
May 2016 on 10.11.5 El Capitan,
July 2016 on 10.12.6 Sierra,
March 2018 on 10.13.3 High Sierra,
January 2019 on Mojave 10.14.
September 2021 on Big Sur (macOS 11.6).
February 2023 on Ventura (macOS 13.2.1).
March 2024 on Sonoma (macOS 14.4) -- had problems.

Summary

In a Terminal window,

  1. Set ARCHFLAGS in shell environment to your hardware architecture.
  2. Install or update Homebrew.
  3. Install or update Perl in Homebrew; link /usr/local/bin/perl to it.
  4. Ensure that Perl shebang lines in ~/lib are #!/usr/local/bin/perl.
  5. Set PERL5LIB in your shell environment.
  6. If you use MySQL, dump database to a temp file; install or update MySQL; reload database.
  7. Install libmaxminddb if wanted.
  8. Install CPAN modules.
  9. If you use MySQL, start MySQL server; install CPAN module DBD::mysql; restore .my.cnf.
  10. test

Prerequisites

Interaction with Perl from Homebrew

As of Ventura, to install modules from Homebrew, Homebrew will require and install a recent version of Perl into /opt/local/bin/perl. Part of this installation will

Before installing CPAN modules, you need to change your shell environment to point to the new Perl.

export PERL5LIB="$HOME/bin:/usr/local/Cellar/perl/5.36.0/lib/perl5/site_perl/5.36.0"

For compatibility, it is best to make the "shebang" line of Perl programs point to something all systems have, and then have that be a symlink to the new Perl installed by Homebrew. This way, you can move Perl programs between macOS and Unix without changing them. Make the first line of all Perl programs be 

  #!/usr/local/bin/perl

And then make that link point to the Homebrew version 

  sudo ln -s /opt/homebrew/bin/perl /usr/local/bin/perl

(On recent versions of macOS, /usr/bin cannot be changed and may not contain an up-to-date Perl.) When Apple releases an update to the OS, check to make sure that /usr/local/bin/perl is still a link to Perl and fix it if necessary.

If necessary, make /usr/local/bin/perl on your Unix systems point to the system Perl.

Install Process

TODO: use and document autobundle.

This installation process takes a long time.

Make sure you have the correct version of Xcode for your OS installed. (It is free from Apple.) It is no longer sufficient to just do xcode-select --install. You need the full Xcode to find the right libraries and header files.

After checking my environment, I did sudo -H cpan. CPAN started and asked if I wanted it to set as many default options as possible. I said yes, and it appeared to work. Then CPAN asked if it should choose the best mirror sites for me. Say NO and set them yourself. Executing r worked, and showed many modules out of date.

I was then able to install modules from the command line. I did install CPAN and reload cpan to upgrade the CPAN module to the latest version, and this worked.

Next I tried install LWP and this failed with the message "Can't locate object method 'requirements_for_module'". This was a symptom of not setting PERL5LIB correctly.

I made a shell script with a line sudo cpan install THING for all these modules:

Modules I Install

Perl support for MySQL

If you call MySQL functions from Perl, install MySQL on your Mac (I use homebrew) before installing the Perl module. To install DBD::mysql, the MySQL server must be started. If you have a .my.cnf file in your home directory, rename it temporarily. Then execute sudo -H cpan. If you have already set the root password for MySQL, do o conf makepl_arg "--testuser=root --testpass=xxxxxxxx", before doing install DBD::mysql and o conf makepl_arg "" afterward. After installing DBD::mysql,

Experience Installing on Sonoma

In 2024, when I upgraded to macOS Sonoma 14.4, I thought everything was fine. My CPAN modules from Ventura worked. Then I unwisely typed brew update ImageMagick: this caused lots of programs to be upgraded, including Perl. I updated $HOME/thvv.bvars to change PERL5LIB to refer to the new version of Perl, 5.38.2.

Then I typed expandfile and got the error that Perl module LWP was missing... as were all other CPAN modules. (I have seen this behavior before after updating Perl.) So I did a long sequence of installs to reinstall all the CPAN modules I use, as listed above, using cpanm. Installing MaxMind::DB::Reader::XS failed due to missing prerequisites... I fixed that by installing Math::Int128 and MaxMind::DB::Metadata and MaxMind::DB::Reader::Role::HasMetadata.

dyld error installing DBD::mysql on Sonoma (fixed)

Then I hit a real snag: installing DBD::mysql failed with a compilation error. The error said "symbol '_mysql_affected_rows' could not be found in the flat address space."

I couldn't get past this error. I posted my woes on stackoverflow.com and Håkon Hægland posted a comment that led me to me the answer.. and it worked again. The story is in https://stackoverflow.com/questions/78202743/installing-cpan-module-dbdmysql-on-macos-sonoma-fails-with-symbol-not-found/78208035#78208035 A thousand thanks to Håkon.

Experience Installing on Ventura

In 2023, I switched my hardware to Apple Silicon, and installed Ventura. so I had to change my ARCHFLAGS and reinstall or recompile to get C programs compiled for -arch arm64. I also had to re-install MySQL via Homebrew to get native binaries. All CPAN modules installed without errors.

Experience Installing on Big Sur

dyld error installing DBD::mysql on Big Sur (fixed)

Installing DBD::mysql on Big Sur, I got the same error dyld: Library not loaded: @rpath/libmysqlclient.21.dylib. As suggested by Google, I tried sudo install_name_tool -id /usr/local/mysql-8.0.26-macos11-x86_64/lib/libmysqlclient.21.dylib /usr/local/mysql-8.0.26-macos11-x86_64/lib/libmysqlclient.21.dylib and got an error message /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: warning: changes being made to the file will invalidate the code signature in: /usr/local/mysql-8.0.26-macos11-x86_64/lib/libmysqlclient.21.dylib

Then I got the errors dyld: Library not loaded: libssl.1.1.dylib and dyld: Library not loaded: libcrypto.1.1.dylib, so I did 

  ln -s /usr/local/opt/openssl/lib/libssl.1.1.dylib /usr/local/lib/libssl.1.1.dylib
  ln -s /usr/local/opt/openssl/lib/libcrypto.1.1.dylib /usr/local/lib/libcrypto.1.1.dylib

Then, I got an error 186 running the CPAN tests. This means "unable to write." I checked: the disk was not full. I looked at the Data directory for MySQL, and all the data files in that directory were owned by my userid instead of mysql:mysql. Not sure what I did to cause this... but fixing it solved the problem.

Finally DBD::mysql succeeded.

Experience Installing on Mojave

Macports installed Perl 5.34, newer than the Apple-provided 5.18. I ensured that /usr/local/bin/perl linked to /opt/local/bin/perl and that PER5LIB referenced /Users/thvv/bin:/opt/local/lib/perl5/5.34. (Later installs brought in Perl 5.28.)

I encountered some problems with individual modules.

Wrong version of XCode

Make sure you have the right version of Xcode installed, or modules will not build. I didn't understand that when I was offered Xcode 11.1 for macOS 10.15 Catalina, that installing it would prevent building for macOS 10.14 Mojave. This mistake also prevented the compilation of libmaxminddb. I had to reinstall the old Xcode.

MaxMind and libmaxminddb and autoconf

Before you install MaxMind::DB::Reader::XS, you must install libmaxminddb from https://github.com/maxmind/libmaxminddb or from Homebrew. Then you need to copy /usr/local/include/maxminddb.h and /usr/local/include/maxminddb_config.h into /opt/local/include/ before installing the module, if your Perl was installed by MacPorts.

On Mojave, trying to install GeoIP2::Database::Reader failed when it tried to install the prerequisite NetAddr::IP and got a mysterious error 255. Finally I did a force install, since the errors were in IPV6 addresses, which I am not using. This problem didn't occur on Big Sur.

(July 2021) I had problems installing libmaxminddb on Mojave. The code shipped without the configure script described in the documentation. There is a configure.ac file: but executing autoconf leads to a Perl error in autom4te. This problem didn't occur on Big Sur.

dyld error installing DBD::mysql on Mojave (fixed)

Installing DBD::mysql on Mojave, I got the error dyld: Library not loaded: @rpath/libmysqlclient.21.dylib. As suggested by Google, I tried sudo install_name_tool -id /usr/local/mysql-8.0.14-macos10.14-x86_64/lib/libmysqlclient.21.dylib /usr/local/mysql-8.0.14-macos10.14-x86_64/lib/libmysqlclient.21.dylib to fix this and got further with cpan.

Then I got the errors dyld: Library not loaded: libssl.1.0.0.dylib and dyld: Library not loaded: libcrypto.1.0.0.dylib. These libraries are loaded in /opt/local/lib. So I tried sudo ln -s /opt/local/lib/libcrypto.1.0.0.dylib /usr/local/lib/libcrypto.1.0.0.dylib and sudo ln -s /opt/local/lib/libssl.1.0.0.dylib /usr/local/lib/libssl.1.0.0.dylib and it worked. Perhaps instead of using install_name_tool I could have just done sudo ln -s /usr/local/mysql-8.0.14-macos10.14-x86_64/lib/libmysqlclient.21.dylib /usr/local/lib/libmysqlclient.21.dylib.

I left the ugly link in /opt/local/lib.

SSL library versions on Mojave (fixed)

I tried this same sequence of installs about 8 months later. Everything seemed to work up until DBD::mysql. It got the same error message and I tried the symlinks above. But libssl.1.0.0.dylib and libssl.1.0.0.dylib were not installed in /usr/local/lib/. Instead I had libssl.1.1.dylib and libssl.1.1.dylib in /usr/local/lib/. In other words, macports had installed a newer version of libcrypto and libssl than DBD::mysql wanted. So, with no other ideas, I did sudo ln -s /opt/local/lib/libcrypto.1.1.dylib /usr/local/lib/libcrypto.1.0.0.dylib and sudo ln -s /opt/local/lib/libssl.1.1.dylib /usr/local/lib/libssl.1.0.0.dylib and the install completed.

I thought it was working, and most functions worked, but my test that invoked LWP::Simple::get failed with 

    dyld: lazy symbol binding failed: Symbol not found: _SSL_library_init
      Referenced from: /opt/local/lib/perl5/site_perl/5.34/darwin-thread-multi-2level/auto/Net/SSLeay/SSLeay.bundle
      Expected in: /opt/local/lib/libssl.1.0.0.dylib

    dyld: Symbol not found: _SSL_library_init
      Referenced from: /opt/local/lib/perl5/site_perl/5.34/darwin-thread-multi-2level/auto/Net/SSLeay/SSLeay.bundle
      Expected in: /opt/local/lib/libssl.1.0.0.dylib

    test.sh: line 42: 31115 Abort trap: 6           $EXPAND $CONFIG testbindcsv.htmt

Installing version 5.34 of Perl fixed these problems. Maybe I don't need the links for libcrypto.1.0.0.dylib and libssl.1.0.0.dylib any more.

Experience Installing on High Sierra and Sierra

I installed Perl 5.26 using Macports, newer than the Apple-provided 5.18.

(Gotcha) Perl 5.22 and later emit complaints about some regular expressions that 5.18 accepted happily. I had to change all my code to alter /{{..../ to /\{\{..../ to put a backslash in front of open braces. This is backward compatible.

Experience Installing on El Capitan

(Another gotcha) CPAN on El Capitan failed installing DBD::mysql with the message 

  # Failed test 'dbh insert id 0 == max(id) 2 in dbd_mysql_t31'
  # at t/31insertid.t line 58.
  # got: 0
  # expected: 2
  # Looks like you failed 1 test of 18.

I had to do force install DBD::mysql and that seemed to work.

Experience Installing on Yosemite

Installing OSX 10.10.5 broke my Perl setup. I had set PERL5LIB to $HOME/bin:/opt/local/lib/perl5/site_perl/5.16.3, but 10.10.5 installed Perl 5.18 as the system Perl. (It was a mistake to say /site_perl, should have omitted this.) Access to MySQL from Perl started failing, saying that the 5.16 library was incompatible with the 5.18 Perl loader. I set PERL5LIB to $HOME/bin:/Library/Perl/5.18/, trying to avoid using the MacPorts Perl, and made a symlink to the new system Perl in my /bin directory with ln -s /usr/local/bin/perl. Then I did sudo -H /usr/bin/cpan to install a bunch of stuff into the system Perl lib, such as DBI. CPAN still failed installing DBD::mysql, even after setting the CPAN config using o conf makepl_arg "--testuser=aaa --testpass=bbb". I searched the Web, which suggested sudo ln -s /usr/local/mysql/lib/libmysqlclient.18.dylib /usr/local/lib/libmysqlclient.18.dylib and this allowed me to install DBD::mysql and run Perl MySQL database accesses. (This problem did not occur on later releases.)

Experience Installing on OSX Snow Leopard and Mavericks

When I executed install DBD::mysql in CPAN on Snow Leopard, the install failed, saying it could not find libmysqlclient.18.dylib. (This problem did not occur on Mountain Lion, but it did on Mavericks.) I Googled and found http://probably.co.uk/problems-installing-dbdmysql-on-os-x-snow-leopard.html . It suggested 

 sudo install_name_tool -id /usr/local/mysql/lib/libmysqlclient.18.dylib\
   /usr/local/mysql/lib/libmysqlclient.18.dylib

I did this and re-tried install DBD::mysql, and it succeeded. This problem did not occur on Yosemite and El Capitan.

Experience Installing on Linux

Experience on Ubuntu Linux

I upgraded a Ubuntu server to 16.04 Xenial. My CPAN modules were not there after the upgrade. So I installed them using the same process. But DBD::mysql failed to install, with the message 

  could not install CPAN module DBD::mysql

  Configuring D/DV/DVEEDEN/DBD-mysql-4.050.tar.gz with Makefile.PL
  Can't exec "mysql_config": No such file or directory at Makefile.PL line 89.

I searched Google for the answer and found several wrong answers, and then the solution. The mysql_config command wasn't installed. I had to do 

  sudo apt-get install libdbd-mysql-perl
  sudo apt-get install libmysqlclient-dev

I did this and re-tried install DBD::mysql, and it succeeded.

Experience on Fedora Linux

On Fedora 31 Linux, install both mysql-community-server and community-mysql-devel before installing DBD::mysql, in order to get mysql_config.

On Fedora 31 Linux, installing NET::DNS fails with errors about a missing system include file. Still researching.

How it works internally on the Mac with MacPorts

I am writing this down so I remember what I spent some time figuring out. Many thanks to the folks at StackOverflow who helped me.

Installing a version of Perl with MacPorts will set up a symlink /opt/local/bin/perl to point to perl5.34 or whatever the latest installed version is. Old versions of perl, perldoc, etc will be left lying around and could be cleaned up, but they are harmless.

Installing a version of Perl with MacPorts will also load a library subtree for various Perl releases into /opt/local/lib/perl5, e.g. /opt/local/lib/perl5/5.34. You may have several old versions, again harmless. There are also two other directories, site_perl and vendor_perl, also containing subtrees for Perl releases, e.g. /opt/local/lib/perl5/site_perl/5.34. The site_perl directory is where module are installed by CPAN. If you specify PERL5LIB=$HOME/bin:/opt/local/lib/perl5/5.34/opt/local/lib/, then Perl will set @INC to 

    /Users/thvv/bin
    /opt/local/lib/perl5/5.34/darwin-thread-multi-2level
    /opt/local/lib/perl5/5.34
    /opt/local/lib/perl5/site_perl/5.34/darwin-thread-multi-2level
    /opt/local/lib/perl5/site_perl/5.34
    /opt/local/lib/perl5/vendor_perl/5.34/darwin-thread-multi-2level
    /opt/local/lib/perl5/vendor_perl/5.34
    /opt/local/lib/perl5/5.34/darwin-thread-multi-2level
    /opt/local/lib/perl5/5.34

In each library subtree you will find .pm files for Perl modules like XSloader.pm, subdirectories for groups of modules like Math/, and special directories darwin-thread-multi-2level for Mac architecture dependent code. Some of the files in darwin-thread-multi-2level invoke XSloader to load in a compiled C version from a .bundle file in darwin-thread-multi-2level/auto.

Perl will derive values for PERL_LOCAL_LIB_ROOT, PERL_MB_OPT, and PERL_MM_OPT from the directory named in PERL5LIB. You don't need to set them yourself and probably should not, to avoid errors.

Execute the command perl -V to check all the configuration values being used by Perl.

Home | FAQ © 2010-2024, Tom Van Vleck updated 2024-03-26 10:22