CPAN on OS X

CPAN, the Comprehensive Perl Archive Network is a free community provided library of Perl programs. My program expandfile, which I use to generate web pages, uses CPAN modules.

Installing CPAN

Here are instructions for upgrading 64-bit CPAN, tested April 2014 on 10.9.2 Mavericks,
July 2015 on 10.10.4 Yosemite,
May 2016 on 10.11.5 El Capitan,
July 2016 on 10.12.6 Sierra,
March 2018 on 10.13.3 High Sierra,
January 2019 on Mojave 10.14.
September 2021 on Big Sur (macOS 11.6).

Prerequisites

Interaction with Perl from Homebrew

To install modules from Homebrew, Homebrew will require and install a recent version of Perl into /opt/local/bin/perl. (Apple ships an old (2013) version 5.18 of Perl, but will stop shipping Perl with the OS in Big Sur.) Before installing CPAN modules, you need to change your startup environment to point to the new Perl. For example, the latest version as of 07/26/21 is 5.34.

    export PERL5LIB=/Users/thvv/bin:/opt/local/lib/perl5/5.34

Change the "shebang" line #!/usr/bin/perl of Perl programs in $HOME/bin so they find the new version of Perl.

      #!/usr/local/bin/perl

Do this so that these programs won't have to change again, and are portable to other Macs and Unix. (On recent versions of macOS, /usr/bin cannot be changed and may not contain a usable Perl.)

When Apple releases an update to the OS, check to make sure that /usr/local/bin/perl is still a link to Perl and fix it if necessary.

Make similar links on any Unix systems you use your Perl programs on.

Install Process

TODO: use and document autobundle.

This installation process takes a long time.

Make sure you have the correct version of Xcode for your OS installed. (It is free from Apple.) It is no longer sufficient to just do xcode-select. You need the full Xcode to find the right libraries and header files.

After checking my environment, I did sudo -H cpan. CPAN started and asked if I wanted it to set as many default options as possible. I said yes, and it appeared to work. Then CPAN asked if it should choose the best mirror sites for me. Say NO and set them yourself.. Executing r worked, and showed many modules out of date.

I was then able to install modules from the command line. I did install CPAN and reload cpan to upgrade CPAN to the latest version, and this worked.

Next I tried install LWP and this failed with the message "Can't locate object method 'requirements_for_module'". This was a symptom of not setting PERL5LIB.

I made a shell script with a line sudo install THING for all these modules:

Modules I Install

To install DBD::mysql, first start the MySQL server. If you have already set the root password for MySQL, do o conf makepl_arg "--testuser=root --testpass=xxxxxxxx", before doing install DBD::mysql and o conf makepl_arg "" afterward. After installing DBD,

Installing on Monterey

Haven't tried this version of macOS yet.

dyld error installing DBD::mysql on Big Sur (fixed)

Installing DBD::mysql on Big Sur, I got the same error dyld: Library not loaded: @rpath/libmysqlclient.21.dylib. As suggested by Google, I tried sudo install_name_tool -id /usr/local/mysql-8.0.26-macos11-x86_64/lib/libmysqlclient.21.dylib /usr/local/mysql-8.0.26-macos11-x86_64/lib/libmysqlclient.21.dylib and got an error message /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: warning: changes being made to the file will invalidate the code signature in: /usr/local/mysql-8.0.26-macos11-x86_64/lib/libmysqlclient.21.dylib

Then I got the errors dyld: Library not loaded: libssl.1.1.dylib and dyld: Library not loaded: libcrypto.1.1.dylib, so I did

  ln -s /usr/local/opt/openssl/lib/libssl.1.1.dylib /usr/local/lib/libssl.1.1.dylib
  ln -s /usr/local/opt/openssl/lib/libcrypto.1.1.dylib /usr/local/lib/libcrypto.1.1.dylib

Then, I got an error 186 running the CPAN tests. This means "unable to write." I checked: the disk was not full. I looked at the Data directory for MySQL, and all the data files in that directory were owned by my userid instead of mysql:mysql. Not sure what I did to cause this... but fixing it solved the problem.

Finally DBD::mysql succeeded.

Installing on Mojave

Macports installed Perl 5.34, newer than the Apple-provided 5.18. I ensured that /usr/local/bin/perl linked to /opt/local/bin/perl and that PER5LIB referenced /Users/thvv/bin:/opt/local/lib/perl5/5.34. (Later installs brought in Perl 5.28.)

I encountered some problems with individual modules.

Wrong version of XCode

Make sure you have the right version of Xcode installed, or modules will not build. I didn't understand that when I was offered Xcode 11.1 for macOS 10.15 Catalina, that installing it would prevent building for macOS 10.14 Mojave. This mistake also prevented the compilation of libmaxminddb. I had to reinstall the old Xcode.

MaxMind and libmaxminddb and autoconf

Before you install MaxMind::DB::Reader::XS, you must install libmaxminddb from https://github.com/maxmind/libmaxminddb or from Homebrew. Then you need to copy /usr/local/include/maxminddb.h and /usr/local/include/maxminddb_config.h into /opt/local/include/ before installing the module, if your Perl was installed by MacPorts.

On Mojave, trying to install GeoIP2::Database::Reader failed when it tried to install the prerequisite NetAddr::IP and got a mysterious error 255. Finally I did a force install, since the errors were in IPV6 addresses, which I am not using. This problem didn't occur on Big Sur.

(July 2021) I had problems installing libmaxminddb on Mojave. The code shipped without the configure script described in the documentation. There is a configure.ac file: but executing autoconf leads to a Perl error in autom4te. This problem didn't occur on Big Sur.

dyld error installing DBD::mysql on Mojave (fixed)

Installing DBD::mysql on Mojave, I got the error dyld: Library not loaded: @rpath/libmysqlclient.21.dylib. As suggested by Google, I tried sudo install_name_tool -id /usr/local/mysql-8.0.14-macos10.14-x86_64/lib/libmysqlclient.21.dylib /usr/local/mysql-8.0.14-macos10.14-x86_64/lib/libmysqlclient.21.dylib to fix this and got further with cpan.

Then I got the errors dyld: Library not loaded: libssl.1.0.0.dylib and dyld: Library not loaded: libcrypto.1.0.0.dylib. These libraries are loaded in /opt/local/lib. So I tried sudo ln -s /opt/local/lib/libcrypto.1.0.0.dylib /usr/local/lib/libcrypto.1.0.0.dylib and sudo ln -s /opt/local/lib/libssl.1.0.0.dylib /usr/local/lib/libssl.1.0.0.dylib and it worked. Perhaps instead of using install_name_tool I could have just done sudo ln -s /usr/local/mysql-8.0.14-macos10.14-x86_64/lib/libmysqlclient.21.dylib /usr/local/lib/libmysqlclient.21.dylib.

I left the ugly link in /opt/local/lib.

SSL library versions on Mojave (fixed)

I tried this same sequence of installs about 8 months later. Everything seemed to work up until DBD::mysql. It got the same error message and I tried the symlinks above. But libssl.1.0.0.dylib and libssl.1.0.0.dylib were not installed in /usr/local/lib/. Instead I had libssl.1.1.dylib and libssl.1.1.dylib in /usr/local/lib/. In other words, macports had installed a newer version of libcrypto and libssl than DBD::mysql wanted. So, with no other ideas, I did sudo ln -s /opt/local/lib/libcrypto.1.1.dylib /usr/local/lib/libcrypto.1.0.0.dylib and sudo ln -s /opt/local/lib/libssl.1.1.dylib /usr/local/lib/libssl.1.0.0.dylib and the install completed.

I thought it was working, and most functions worked, but my test that invoked LWP::Simple::get failed with

    dyld: lazy symbol binding failed: Symbol not found: _SSL_library_init
      Referenced from: /opt/local/lib/perl5/site_perl/5.34/darwin-thread-multi-2level/auto/Net/SSLeay/SSLeay.bundle
      Expected in: /opt/local/lib/libssl.1.0.0.dylib

    dyld: Symbol not found: _SSL_library_init
      Referenced from: /opt/local/lib/perl5/site_perl/5.34/darwin-thread-multi-2level/auto/Net/SSLeay/SSLeay.bundle
      Expected in: /opt/local/lib/libssl.1.0.0.dylib

    test.sh: line 42: 31115 Abort trap: 6           $EXPAND $CONFIG testbindcsv.htmt

Installing version 5.34 of Perl fixed these problems. Maybe I don't need the links for libcrypto.1.0.0.dylib and libssl.1.0.0.dylib any more.

Installing on High Sierra and Sierra

I installed Perl 5.26 using Macports, newer than the Apple-provided 5.18.

(Gotcha) Perl 5.22 and later emit complaints about some regular expressions that 5.18 accepted happily. I had to change all my code to alter /{{..../ to /\{\{..../ to put a backslash in front of open braces. This is backward compatible.

Installing on El Capitan

(Another gotcha) CPAN on El Capitan failed installing DBD::mysql with the message

  # Failed test 'dbh insert id 0 == max(id) 2 in dbd_mysql_t31'
  # at t/31insertid.t line 58.
  # got: 0
  # expected: 2
  # Looks like you failed 1 test of 18.

I had to do force install DBD::mysql and that seemed to work.

Installing on Yosemite

Installing OSX 10.10.5 broke my Perl setup. I had set PERL5LIB to $HOME/bin:/opt/local/lib/perl5/site_perl/5.16.3, but 10.10.5 installed Perl 5.18 as the system Perl. (It was a mistake to say /site_perl, should have omitted this.) Access to MySQL from Perl started failing, saying that the 5.16 library was incompatible with the 5.18 Perl loader. I set PERL5LIB to $HOME/bin:/Library/Perl/5.18/, trying to avoid using the MacPorts Perl, and made a symlink to the new system Perl in my /bin directory with ln -s /usr/local/bin/perl. Then I did sudo -H /usr/bin/cpan to install a bunch of stuff into the system Perl lib, such as DBI. CPAN still failed installing DBD::mysql, even after setting the CPAN config using o conf makepl_arg "--testuser=aaa --testpass=bbb". I searched the Web, which suggested sudo ln -s /usr/local/mysql/lib/libmysqlclient.18.dylib /usr/local/lib/libmysqlclient.18.dylib and this allowed me to install DBD::mysql and run Perl MySQL database accesses. (This problem did not occur on later releases.)

Installing on OSX Snow Leopard and Mavericks

When I executed install DBD::mysql in CPAN on Snow Leopard, the install failed, saying it could not find libmysqlclient.18.dylib. (This problem did not occur on Mountain Lion, but it did on Mavericks.) I Googled and found http://probably.co.uk/problems-installing-dbdmysql-on-os-x-snow-leopard.html . It suggested

 sudo install_name_tool -id /usr/local/mysql/lib/libmysqlclient.18.dylib\
   /usr/local/mysql/lib/libmysqlclient.18.dylib

I did this and re-tried install DBD::mysql, and it succeeded. This problem did not occur on Yosemite and El Capitan.

Installing on Linux

Ubuntu

I upgraded a Ubuntu server to 16.04 Xenial. My CPAN modules were not there after the upgrade. So I installed them using the same process. But DBD::mysql failed to install, with the message

  could not install CPAN module DBD::mysql

  Configuring D/DV/DVEEDEN/DBD-mysql-4.050.tar.gz with Makefile.PL
  Can't exec "mysql_config": No such file or directory at Makefile.PL line 89.

I searched Google for the answer and found several wrong answers, and then the solution. The mysql_config command wasn't installed. I had to do

  sudo apt-get install libdbd-mysql-perl
  sudo apt-get install libmysqlclient-dev

I did this and re-tried install DBD::mysql, and it succeeded.

Fedora

On Fedora 31 Linux, install both mysql-community-server and community-mysql-devel before installing DBD::mysql, in order to get mysql_config.

On Fedora 31 Linux, installing NET::DNS fails with errors about a missing system include file. Still researching.

How it works internally on the Mac

I am writing this down so I remember what I spent some time figuring out. Many thanks to the folks at StackOverflow who helped me.

Installing a version of Perl with MacPorts will set up a symlink /opt/local/bin/perl to point to perl5.34 or whatever the latest installed version is. Old versions of perl, perldoc, etc will be left lying around and could be cleaned up, but they are harmless.

Installing a version of Perl with MacPorts will also load a library subtree for various Perl releases into /opt/local/lib/perl5, e.g. /opt/local/lib/perl5/5.34. You may have several old versions, again harmless. There are also two other directories, site_perl and vendor_perl, also containing subtrees for Perl releases, e.g. /opt/local/lib/perl5/site_perl/5.34. The site_perl directory is where module are installed by CPAN. If you specify PERL5LIB=$HOME/bin:/opt/local/lib/perl5/5.34/opt/local/lib/, then Perl will set @INC to

    /Users/thvv/bin
    /opt/local/lib/perl5/5.34/darwin-thread-multi-2level
    /opt/local/lib/perl5/5.34
    /opt/local/lib/perl5/site_perl/5.34/darwin-thread-multi-2level
    /opt/local/lib/perl5/site_perl/5.34
    /opt/local/lib/perl5/vendor_perl/5.34/darwin-thread-multi-2level
    /opt/local/lib/perl5/vendor_perl/5.34
    /opt/local/lib/perl5/5.34/darwin-thread-multi-2level
    /opt/local/lib/perl5/5.34

In each library subtree you will find .pm files for Perl modules like XSloader.pm, subdirectories for groups of modules like Math/, and special directories darwin-thread-multi-2level for Mac architecture dependent code. Some of the files in darwin-thread-multi-2level invoke XSloader to load in a compiled C version from a .bundle file in darwin-thread-multi-2level/auto.

Perl will derive values for PERL_LOCAL_LIB_ROOT, PERL_MB_OPT, and PERL_MM_OPT from the directory named in PERL5LIB. You don't need to set them yourself and probably should not, to avoid errors.

Execute the command perl -V to check all the configuration values being used by Perl.

Home | FAQ © 2010-2021, Tom Van Vleck updated 2021-10-16 15:35