Vagrant/continuous integration/dev machine with fixed third-party software/packages.
If you have a continuous integration server running or some kind of a dev/staging machine for your (software) release management you might considered already vagrant to ease that process. Especially, when you deal with VMs and software releases you usually want that each machine (dev, staging, live) have the exact same services/libs/package installed. Even better with automatically provisioning of your third-party dependencies/packages of that server.
To achieve that goal you have several ways. The hard way would be obviously to download all source packages (php/apache for example) and compile it on the initialized bootstrap of your VM. Beside the fact that this is a pretty time-consuming it’s also not that easy as you have to have all dependencies installed as well to get the compiling running.
The easier way is to install your packages and needed services via the integrated package manager of your (linux) distribution. I talk now here about Debian Package Manager (dpkg, which is also used Ubuntu).
apt-get has the ability to talk not only to a remote repository but also to a local one.
This gives us the possibility to store all installed
*.deb files during the bootstrap in our own (git) repository
and use those packages instead of talking to Debian’s official apt-get repositories.
The advantage of storing all debian files to your repository is that you’ve then the ability to jump in the history
of your server configuration. This is pretty cool, especially when you want to provision a default (dev) VM for all of your
developers. Also a upgrade of some packages is then a ease as you only have to push a new debian package (.deb) to your
git repository and all other can just pull one change instead of download a huge ISO again and again.
So, advantage of this method is:
- Have all third-party dependencies of your VM versioned.
- Provision is way faster as only changes are received not the big/complete VM ISO.
- No internet connection required.
- Always the same versions.
Note: You might thought already about a local package version and installation with
dpkg. This method has the big
disadvantage that you have to resolve all dependencies on your own. That is very time consuming and frustrating,
especially when you install a package with many small dependencies. You have to take care here about the installation
order as well.
So, the only elegant way is to use the well known
apt-get tool. The basic procedure is like this:
- Install all packages with a naked installation.
- Since all packages that have been installed through
apt-getare cached in
/var/cache/apt/archives/you should move or copy all those packages to a own directory.
- Use a script (see below) to create a
- Update your
/etc/apt/sources.listto point to your local repository.
- Next bootstrap is way faster and offline-ready.
Install all needed packages
In the initial step you have to install all packages in the regular way. Connect your machine to the internet
apt-get install <packageName> as usual. If you machine is “dirty” (not a fresh installation), you have
probably already some packages in
/var/cache/apt/archives/. Make sure those packages a really required.
Best way is to use a really fresh net installation. (like those of vagrant)
Create a own repository
In vagrant you have per default the
/vagrant directory available. Since it’s a “mount” to your local filesystem to the
directory where your Vagrantfile is located you’ve probably already a git repository there. If not, you should create
a git repository and a sub-directory called
Copy now all your
*.deb files from
$ cp /var/cache/apt/archives/*.deb ./apt-repository/
To build the “index” file that is used in the debian repository to know which packages are available you can use following:
$ cd ./apt-repository/ $ dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
Always when you add a new debian package (
.deb) file to our
./apt-repository/ directory you have to run this command again.
Use our own repository
So, the process above is basically only needed in the very first step of the setup of your machine.
You have now to say
apt-get that it should use our own repository. Again, since in vagrant the
/vagrant is available
per default you don’t have to do something special to have all packages available.
Create at first a new
sources.list file in your repository
./sources.list and add following content:
deb file:/vagrant/apt-repository ./
In your vagrant bootstrap (
(usually fired in
config.vm.provision :shell, :path => "bootstrap.sh")
if [ ! -f /etc/apt/sources.list.old ] then mv /etc/apt/sources.list /etc/apt/sources.list.old cp /vagrant/sources.list /etc/apt/sources.list fi apt-get update
Make sure this lines of bash code are placed at the very first. All following
apt-get install calls use then our local
The trick is now, each time you want to upgrade a package to a newer version you just have to move your
apt-get update again. Then just install the new packages as usual via
apt-get install ..., move then
.deb files to your
./apt-repository and fire the
dpkg-scanpackages from above again.
After committing and pushing your changes to the git repository all your developers or your continuous integration server is able to use automatically the newer version.
You can find more information about the debian repository structure in the manual Repository Howto.