The Leading Educational Resource for IT Professionals

TechTip: IFS Containers Part 3, Installing Open Source from the Internet

by Victoria Mack December 09, 2016 0 Comments

One of the core tenets of open source is freedom to obtain and use software. Now it's time to see that benefit in action.

In the first article of this series, we learned about how IFS Containers could be manually created by copying things into a new directory and then running the chroot command against that directory. This was a laborious process that would cause the technology to not be used unless some amount of automation was introduced. In the second article, I introduced the IBM i Chroot (ibmichroot) project, which addresses the automation of creating IFS Containers in a matter of minutes and even more important, uniformity. In this article, we will navigate through the other core feature of the IBM i Chroot project: package installation.

By “package installation,” I mean the process of obtaining software from the Internet and installing it in PASE. This is where one of the core tenets of open source comes into play: the ability to freely obtain software for download or compilation from others. A first question people might have is "What packages are we talking about?" Well, when I set up IFS Containers, there are a handful of packages I always include. I've listed them below.

  • JoeA server-side editor that allows me to quickly edit files in the IFS from a shell session
  • WgetCommand to access URLs; great for quickly downloading things from the Internet
  • ZshThe zsh shell is an alternative to bash, though both work well and I use both

As you can see, my usage of packages involves tools and commands, but they extend much further than that to include things like web servers, programming languages, and much more.

The IBM i Chroot project has two approaches to obtaining and installing packages. The first one is what's been in there since the beginning of the project: obtaining packages from perzl.org. The second is using the yum command to obtain packages. Using yum is still in an experimental phase, so we won't be discussing that here. You can see the yum progress here.

Let's dive into the perzl.org approach to packages. First, what is perzl.org? It is a site maintained by Michael Perzl; an IBMer who ports Linux packages to AIX. This is very cool for us because PASE is essentially AIX and that means when perzl.org ports a package it can also run on IBM i. You can see the complete list of ported packages here. (It's important to note his site isn't endorsed or supported by IBM and instead is a personal endeavor.)

To install a perzl.org package, we need to run the following commands. Please refer to the second article in this series for how to install IBM i Chroot on your IBM i.

$ cd /QOpenSys/ibmichroot/pkg

$ pkg_setup.sh pkg_perzl_joe-3.7-1.lst

**********************

Live IBM i session (changes made).

**********************

PATH=/QOpenSys/usr/bin:/QOpenSys/usr/sbin:/opt/freeware/bin

LIBPATH=/QOpenSys/usr/lib:/opt/freeware/lib

--12:04:03-- http://www.oss4aix.org/download/RPMS/joe/joe-3.7-1.aix5.1.ppc.rpm

           => `joe-3.7-1.aix5.1.ppc.rpm'

Resolving www.oss4aix.org... 178.254.6.100

Connecting to www.oss4aix.org[178.254.6.100]:80... connected.

HTTP request sent, awaiting response... 200 OK

Length: 342,946 [application/x-rpm]

100%[====================================>] 342,946     406.18K/s

12:04:04 (404.86 KB/s) - `joe-3.7-1.aix5.1.ppc.rpm' saved [342946/342946]

install joe-3.7-1.aix5.1.ppc.rpm ...

failed to stat /QNTC: No such file or directory

joe                         ##################################################

First, we use the cd command to enter the /QOpenSys/ibmichroot/pkg directory so we can operate without fully qualifying other paths in the next command. Next, we run the pkg_setup.sh script and provide the package configuration we would like to install, namely pkg_perzl_joe-3.7-1.lst. Package configuration files are used to determine where a package's files are located on the Internet and also what its dependencies are.

Below are the contents of configuration file pkg_perzl_joe-3.7-1.lst.

# Command:

#   > ./pkg_setup.sh pkg_perzl_joe-3.7-1.lst

# Use:

#   > export PATH=/opt/freeware/bin:$PATH

#   > export LIBPATH=/opt/freeware/lib:$LIBPATH (if error, remove ':$LIBPATH')

#   > any_util (below)

# Contents:

# Other files:

#

# read file

#

:file

#

# rpm list

#

:rpm

http://www.oss4aix.org/download/RPMS/joe/joe-3.7-1.aix5.1.ppc.rpm

The # symbol is considered a comment. The :file and :rpm areas are declaring a named delineation of the file. Specifically, :file is used to declare other package configuration files that should be run before proceeding. This is nice for modularizing package installs. The :rpm section is used to declare what RPMs should be installed to completely install the package. The joe utility is simple and has no dependencies. Look at pkg_perzl_gcc-4.8.3.lst to see a larger example with dependencies.

Before going on, it would be good to describe what an rpm is. Wikipedia defines RPM as:

RPM Package Manager (RPM) (originally Red Hat Package Manager; now a recursive acronym) is a package management system. The name RPM refers to the following: the .rpm file format, files in the .rpm file format, software packaged in such files, and the package manager program itself. RPM was intended primarily for Linux distributions; the file format is the baseline package format of the Linux Standard Base.

Even though it was created for use in Red Hat Linux, RPM is now used in many Linux distributions. It has also been ported to some other operating systems, such as Novell NetWare (as of version 6.5 SP3) and IBM's AIX (as of version 4).

An RPM package can contain an arbitrary set of files. The larger part of RPM files encountered are “binary RPMs” (or BRPMs) containing the compiled version of some software. There are also “source RPMs” (or SRPMs) files containing the source code used to produce a package. These have an appropriate tag in the file header that distinguishes them from normal (B)RPMs, causing them to be extracted to /usr/src on installation. SRPMs customarily carry the file extension “.src.rpm” (.spm on file systems limited to 3 extension characters, e.g. old DOS FAT).

Think of an RPM file as being similar to a *SAVF or licensed program install; it can contain compiled programs, IFS files, and pre/post execution scripts.

At this point, the joe editor is installed and available for use. I usually test a package to make sure it is working after I install it. First, I run the following command to learn that the joe command exists and where it is located.

$ which joe | xargs ls -al

lrwx------   1 aaron   0       52 Aug 31 10:18 /QOpenSys/usr/bin/joe -> ../../opt/freeware/bin/joe

And since joe is an editor, you can test it further by actually editing a file using the following command.

$ joe mynewfile.txt

You should now see a screen like the following. Note that you need to have logged in via an SSH connection. Using CALL QP2TERM won't work with joe.

120916bartell figure1 joe dmu

Figure 1: Joe editor in action on IBM i

Before moving on, I want to share a tip I recently learned. Below are a couple user profiles I recommend you create on your system. These aren't required but will greatly lessen the amount of warning logs created while using the pkg_setup.sh command. You'll notice I didn't have many warning messages relating to a profile named “system” or “bin”; that's because the server I ran the install on already had those profiles.

CRTUSRPRF USRPRF(SYSTEM) PASSWORD(*NONE) USRCLS(*SYSOPR) TEXT('AIX rpm group') SPCAUT(*USRCLS) GID(*GEN) AUT(*ALL)

CRTUSRPRF USRPRF(BIN) PASSWORD(*NONE) USRCLS(*SYSOPR) TEXT('AIX rpm group') SPCAUT(*USRCLS) GID(*GEN) AUT(*ALL)

Up to this point, we haven't really talked about how IFS Containers fit into the picture with pkg_setup.sh. You can install packages inside an IFS Container or in what I call the "base of PASE" (outside an IFS Container, where it will affect everyone). My best practice is to install as little as possible in the base of PASE. For example, I will usually install zsh, joe, and wget in the base of PASE, but I try to stop at that because, again, it affects everyone, which means I could unintentionally break other things that are running in the base of PASE. In contrast, when IFS Containers are used, we can install packages without worry that they'll affect others.

The steps above are the same when installing in an IFS Container. The difference is you first need to put the IBM i Chroot project inside the IFS Container. Why? Recall from the first article where we learned that if you don't put a file into an IFS Container, then it can't be found.

Below are the commands necessary to create an IFS Container, copied verbatim from the second article in this series.

$ mkdir /QOpenSys/ibmichroot_containers

$ cd /QOpenSys/ibmichroot/chroot

$ chroot_setup.sh chroot_minimal.lst /QOpenSys/ibmichroot_containers/c1

The below cp (copy) command will take the ibmichroot directory from the base of PASE and copy it into the same-named directory in the c1 IFS Containter. You can learn about the copy command in the AIX documentation.

$ cp -R /QOpenSys/ibmichroot /QOpenSys/ibmichroot_containers/c1/QOpenSys/.

Now enter the IFS Container using the chroot command, as shown below.

$ chroot /QOpenSys/ibmichroot_containers/c1 /usr/bin/sh

At this point, as previously mentioned, the process is the same as when we installed packages in the base of PASE. Simply cd (change directory) to the location of the pkg_setup.sh script and then run the script while specifying a package configuration.

$ cd /QOpenSys/ibmichroot/pkg

$ pkg_setup.sh pkg_perzl_joe-3.7-1.lst

**********************

Live IBM i session (changes made).

**********************

PATH=/QOpenSys/usr/bin:/QOpenSys/usr/sbin:/opt/freeware/bin

LIBPATH=/QOpenSys/usr/lib:/opt/freeware/lib

setup rpm.rte ...

New volume on rpm.rte:

Cluster 51200 bytes (100 blocks).

   Volume number 1

   Date of backup: Thu Mar 1 09:26:22 2012

   Files backed up by name

   User BUILD

x           0 ./

x         2473 ./lpp_name

x           0 ./usr

x            0 ./usr/lpp

x           0 ./usr/lpp/rpm.rte

x       57382 ./usr/lpp/rpm.rte/liblpp.a

x           0 ./usr/lpp/rpm.rte/inst_root

x         8610 ./usr/lpp/rpm.rte/inst_root/liblpp.a

x           0 ./usr/lpp/rpm.rte/inst_root/etc/rpm

x         1147 ./usr/lpp/rpm.rte/inst_root/etc/info-dir

x           0 ./usr/linux

x           0 ./usr/linux/bin

x           0 ./usr/opt/freeware/bin

x       33713 ./usr/opt/freeware/bin/bunzip2

x       33713 ./usr/opt/freeware/bin/bzcat

x       33713 ./usr/opt/freeware/bin/bzip2

x         9329 ./usr/opt/freeware/bin/bzip2recover

x         410 ./usr/opt/freeware/bin/gendiff

x         3921 ./usr/opt/freeware/bin/gzexe

x       64090 ./usr/opt/freeware/bin/gzip

x       26376 ./usr/opt/freeware/bin/install-info

x       94446 ./usr/opt/freeware/bin/patch

x       74385 ./usr/opt/freeware/bin/rpm

x       19592 ./usr/opt/freeware/bin/rpm2cpio

x         2009 ./usr/opt/freeware/bin/zcmp

x         1342 ./usr/opt/freeware/bin/zgrep

x         1077 ./usr/opt/freeware/bin/zmore

x         3511 ./usr/opt/freeware/bin/znew

x           0 ./usr/opt/freeware/packages

x       13005 ./usr/sbin/rpm_share

x         155 ./usr/bin/rpm

x           0 ./usr/opt/freeware/info

x           0 ./usr/opt/freeware/lib

x       197216 ./usr/opt/freeware/lib/libbz2.a

x       374612 ./usr/opt/freeware/lib/libintl.a

x       46789 ./usr/opt/freeware/lib/libpopt.so.0.0.0

x     1010536 ./usr/opt/freeware/lib/librpm.so.0.0.0

x       200900 ./usr/opt/freeware/lib/librpmbuild.so.0.0.0

x       271724 ./usr/opt/freeware/lib/libz.a

x           0 ./usr/opt/freeware/share/locale

x           0 ./usr/opt/freeware/share/locale/de

x           0 ./usr/opt/freeware/share/locale/de/LC_MESSAGES

x       29327 ./usr/opt/freeware/share/locale/de/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/fr

x           0 ./usr/opt/freeware/share/locale/fr/LC_MESSAGES

x         2933 ./usr/opt/freeware/share/locale/fr/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/pl

x           0 ./usr/opt/freeware/share/locale/pl/LC_MESSAGES

x       63539 ./usr/opt/freeware/share/locale/pl/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/sl

x           0 ./usr/opt/freeware/share/locale/sl/LC_MESSAGES

x       71060 ./usr/opt/freeware/share/locale/sl/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/sv

x           0 ./usr/opt/freeware/share/locale/sv/LC_MESSAGES

x       69145 ./usr/opt/freeware/share/locale/sv/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/cs

x           0 ./usr/opt/freeware/share/locale/cs/LC_MESSAGES

x       37014 ./usr/opt/freeware/share/locale/cs/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/fi

x           0 ./usr/opt/freeware/share/locale/fi/LC_MESSAGES

x       28184 ./usr/opt/freeware/share/locale/fi/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/ja

x           0 ./usr/opt/freeware/share/locale/ja/LC_MESSAGES

x        45951 ./usr/opt/freeware/share/locale/ja/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/pt_BR

x           0 ./usr/opt/freeware/share/locale/pt_BR/LC_MESSAGES

x       17199 ./usr/opt/freeware/share/locale/pt_BR/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/sk

x           0 ./usr/opt/freeware/share/locale/sk/LC_MESSAGES

x         510 ./usr/opt/freeware/share/locale/sk/LC_MESSAGES/popt.mo

x       60406 ./usr/opt/freeware/share/locale/sk/LC_MESSAGES/rpm.mo

x          0 ./usr/opt/freeware/share/locale/sr

x           0 ./usr/opt/freeware/share/locale/sr/LC_MESSAGES

x       24281 ./usr/opt/freeware/share/locale/sr/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/tr

x           0 ./usr/opt/freeware/share/locale/tr/LC_MESSAGES

x       28637 ./usr/opt/freeware/share/locale/tr/LC_MESSAGES/rpm.mo

x           0 ./usr/opt/freeware/share/locale/ro

x           0 ./usr/opt/freeware/share/locale/ro/LC_MESSAGES

x         494 ./usr/opt/freeware/share/locale/ro/LC_MESSAGES/popt.mo

x           0 ./usr/opt/freeware/src

x           0 ./usr/opt/freeware/src/packages

x           0 ./usr/opt/freeware/src/packages/BUILD

x           0 ./usr/opt/freeware/src/packages/RPMS

x          0 ./usr/opt/freeware/src/packages/RPMS/ia64

x           0 ./usr/opt/freeware/src/packages/RPMS/noarch

x           0 ./usr/opt/freeware/src/packages/RPMS/powerpc

x           0 ./usr/opt/freeware/src/packages/RPMS/ppc

x           0 ./usr/opt/freeware/src/packages/SOURCES

x           0 ./usr/opt/freeware/src/packages/SPECS

x           0 ./usr/opt/freeware/src/packages/SRPMS

x           0 ./usr/opt/freeware/lib/rpm

x         1382 ./usr/opt/freeware/lib/rpm/brp-compress

x         374 ./usr/opt/freeware/lib/rpm/brp-redhat

x         372 ./usr/opt/freeware/lib/rpm/brp-strip

x         584 ./usr/opt/freeware/lib/rpm/brp-strip-comment-note

x         422 ./usr/opt/freeware/lib/rpm/check-prereqs

x       33456 ./usr/opt/freeware/lib/rpm/config.guess

x       25283 ./usr/opt/freeware/lib/rpm/config.sub

x         2561 ./usr/opt/freeware/lib/rpm/convertrpmrc.sh

x         4031 ./usr/opt/freeware/lib/rpm/cpanflute

x         2248 ./usr/opt/freeware/lib/rpm/find-lang.sh

x         4188 ./usr/opt/freeware/lib/rpm/find-prov.pl

x       24048 ./usr/opt/freeware/lib/rpm/find-provides

x         267 ./usr/opt/freeware/lib/rpm/find-provides.perl

x         3571 ./usr/opt/freeware/lib/rpm/find-req.pl

x         3106 ./usr/opt/freeware/lib/rpm/find-requires

x         1533 ./usr/opt/freeware/lib/rpm/find-requires.perl

x         2335 ./usr/opt/freeware/lib/rpm/get_magic.pl

x         792 ./usr/opt/freeware/lib/rpm/getpo.sh

x         3206 ./usr/opt/freeware/lib/rpm/http.req

x       18759 ./usr/opt/freeware/lib/rpm/macros

x         3634 ./usr/opt/freeware/lib/rpm/magic.prov

x         2927 ./usr/opt/freeware/lib/rpm/magic.req

x         726 ./usr/opt/freeware/lib/rpm/mkinstalldirs

x         4738 ./usr/opt/freeware/lib/rpm/perl.prov

x         5522 ./usr/opt/freeware/lib/rpm/perl.req

x       23972 ./usr/opt/freeware/lib/rpm/rpmdiff

x       15674 ./usr/opt/freeware/lib/rpm/rpmdiff.cgi

x       58952 ./usr/opt/freeware/lib/rpm/rpmgettext

x         3639 ./usr/opt/freeware/lib/rpm/rpmpopt

x       58952 ./usr/opt/freeware/lib/rpm/rpmputtext

x         8160 ./usr/opt/freeware/lib/rpm/rpmrc

x         1680 ./usr/opt/freeware/lib/rpm/u_pkg.sh

x         8427 ./usr/opt/freeware/lib/rpm/vpkg-provides.sh

x         2429 ./usr/opt/freeware/lib/rpm/vpkg-provides2.sh

x           0 ./usr/opt/freeware/man

x           0 ./usr/opt/freeware/man/man1

x         1610 ./usr/opt/freeware/man/man1/gendiff.1

x           0 ./usr/opt/freeware/man/man8

x       21513 ./usr/opt/freeware/man/man8/rpm.8

x         605 ./usr/opt/freeware/man/man8/rpm2cpio.8

x           0 ./usr/src

x           0 ./usr/lpp/rpm.rte/inst_root/var/lib

x           0 ./usr/lpp/rpm.rte/inst_root/var/opt/freeware

x           0 ./usr/lpp/rpm.rte/inst_root/var/opt/freeware/lib

x           0 ./usr/swlag

x          0 ./usr/swlag/en_US

x       27520 ./usr/swlag/en_US/rpm.rte.la

   total size: 3448054

   files restored: 139

mkdir: cannot create /var.

/var: File exists

ln: /var/opt exists. Specify -f to remove.

setup wget-1.9.1-1.aix5.1.ppc.rpm ...

failed to stat /QNTC: No such file or directory

wget                       ##################################################

GNU Wget 1.9.1

Copyright (C) 2003 Free Software Foundation, Inc.

This program is distributed in the hope that it will be useful,

but WITHOUT ANY WARRANTY; without even the implied warranty of

MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the

GNU General Public License for more details.

Originally written by Hrvoje Niksic <hniksic@xemacs.org>;.

--15:22:27-- http://www.oss4aix.org/download/RPMS/joe/joe-3.7-1.aix5.1.ppc.rpm

           => `joe-3.7-1.aix5.1.ppc.rpm'

Resolving www.oss4aix.org... 178.254.6.100

Connecting to www.oss4aix.org[178.254.6.100]:80... connected.

HTTP request sent, awaiting response... 200 OK

Length: 342,946 [application/x-rpm]

100%[===================================================================================>] 342,946     314.01K/s

15:22:28 (313.50 KB/s) - `joe-3.7-1.aix5.1.ppc.rpm' saved [342946/342946]

install joe-3.7-1.aix5.1.ppc.rpm ...

failed to stat /QNTC: No such file or directory

joe                         ##################################################

That's it! You're now successfully using IFS Containers on IBM i! It's worth noting that the manual steps we took to copy the ibmichroot directory into the c1 IFS Container could be automated by including the copy command in a custom chroot_xxxxx.lst that is for a specific purpose in your business.

On final note, the work of Michael Perzl has been extremely useful for IBM i shops to get up and running with a variety of open-source software packages. What we're now finding is we are outgrowing his efforts. Specifically, what happens if we need to use a package he hasn't yet ported? What do we do if we need a more recent version of a package that isn't yet available on his site? What if the package doesn't install successfully because it's expecting something that's specific to AIX (see Nginx article)? That's why we're venturing into how to create our own RPM files on and for IBM i. This is where the IBM i Chroot efforts with yum come into play. Watch this space, as they say.

If you have any questions or comments, then please comment below or email me at abartell@krengeltech.com.

 

About the Author: Aaron Bartell    




Victoria Mack
Victoria Mack

Author



Also in MC Press Articles

Customer (Citizen) Identity and Access Management

by Graham Williamson June 13, 2017 0 Comments

As a major trend in the IDM sector, consumerization has become easier and exponentially more important. Digital transformation will literally put a significant segment of the SME market out of business and propel a significant number of SMEs to new levels of prosperity.

Continue Reading →

Federated Authentication – there is no Plan B

by Graham Williamson June 05, 2017 0 Comments

Federated authentication is essential for businesses. It's the only way to effectively manage external access to business systems and it's absolutely necessary in order to manage authentication to SaaS apps. if you don't want to expose your identity records to potential compromise.

Continue Reading →

Access Control – RBAC & ABAC

by Graham Williamson May 04, 2017 0 Comments

Access Control is the core of the identity and access management task. Once we have correctly provisioned user data into the enterprise’s identity service we need to leverage it for access control. The vast majority of organizations use role-based access control, but increasingly, access control based on attributes is gaining traction.

Continue Reading →