We use only Linux at work and all our workstations are powered by Ubuntu and the network has more than 20 Ubuntu machines in it. Being admin’s, we are always keen to keep everything updated and stable. Because of this reason, a lot of bandwidth will be eaten on every 6 months, yeah “the distro upgrade” process ð To save the bandwidth and maintain a stable network traffic, we decided to create a local ubuntu repository for Ubuntu and each and every steps to the target has been clearly explained in this post.
The whole process has two stages:
1) Setting up the local ubuntu repository server- ‘reposerver’.
2) Setting up other machines/clients to use our server as the repository source.
STAGE 1:
1) Setting up the local ubuntu repository server: reposerver.
We have to keep a machine dedicated for the local ubuntu repository , lets call this machine as ‘reposerver‘ The main requirement for reposerver is the ‘disk space’, 100GBÂ is recommended.
Here, I have created a separate partition for /var (and mounted it under /media) with 100 GB in size and assigned this space for storing the repo packages. You can also use an external storage device for this purpose. Once the disk space is ready, we need to install two packages on reposerver
apt-mirror , apache2
apt-mirror:
‘apt-mirror’ can easily create a mirror of repository from the Ubuntu server on our local machine(local ubuntu repository). It is a perl-based utility for downloading and mirroring the entire packages of a public Ubuntu repository. To install apt-mirror,
sudo apt-get install apt-mirror
Open the configuration file for apt-mirror
sudo gedit /etc/apt/mirror.list
You will find the below sample configuration file.
############# config ################## # # set base_path /var/spool/apt-mirror # # if you change the base path you must create the directories below with write privileges # # set mirror_path $base_path/mirror # set skel_path $base_path/skel # set var_path $base_path/var # set cleanscript $var_path/clean.sh # set defaultarch <running host architecture> # set postmirror_script $var_path/postmirror.sh set run_postmirror 0 set nthreads 20 set _tilde 0 # ############# end config ############## deb http://archive.ubuntu.com/ubuntu karmic main restricted universe multiverse deb http://archive.ubuntu.com/ubuntu karmic-security main restricted universe multiverse deb http://archive.ubuntu.com/ubuntu karmic-updates main restricted universe multiverse #deb http://archive.ubuntu.com/ubuntu karmic-proposed main restricted universe multiverse #deb http://archive.ubuntu.com/ubuntu karmic-backports main restricted universe multiverse deb-src http://archive.ubuntu.com/ubuntu karmic main restricted universe multiverse deb-src http://archive.ubuntu.com/ubuntu karmic-security main restricted universe multiverse deb-src http://archive.ubuntu.com/ubuntu karmic-updates main restricted universe multiverse #deb-src http://archive.ubuntu.com/ubuntu karmic-proposed main restricted universe multiverse #deb-src http://archive.ubuntu.com/ubuntu karmic-backports main restricted universe multiverse clean http://archive.ubuntu.com/ubuntu
All our machines are now installed with Ubuntu karmic Koala 9.10. We are now going to mirror the repository for the latest release Lucid Lynx so that we can use this to update/upgrade all machines. We need to replace ‘karmic’ with ‘lucid’ on mirror.list. You can use the ‘replace all’ option on gedit. The modified mirror.list file will looks like below.
############# config ################## # set base_path /var/spool/apt-mirror # #set mirror_path $base_path/mirror #set skel_path $base_path/skel #set var_path $base_path/var #set cleanscript $var_path/clean.sh #set defaultarch <running host architecture> #set postmirror_script $var_path/postmirror.sh set run_postmirror 0 set nthreads 20 set _tilde 0 # ############# end config ############## deb http://archive.ubuntu.com/ubuntu lucid main restricted deb http://archive.ubuntu.com/ubuntu lucid-updates main restricted deb http://archive.ubuntu.com/ubuntu lucid universe deb http://archive.ubuntu.com/ubuntu lucid-updates universe deb http://archive.ubuntu.com/ubuntu lucid multiverse deb http://archive.ubuntu.com/ubuntu lucid-updates multiverse deb http://archive.ubuntu.com/ubuntu lucid-backports main restricted universe multiverse deb-src http://archive.ubuntu.com/ubuntu lucid-backports main restricted universe multiverse deb http://archive.canonical.com/ubuntu lucid partner deb-src http://archive.canonical.com/ubuntu lucid partner deb http://security.ubuntu.com/ubuntu lucid-security main restricted deb http://security.ubuntu.com/ubuntu lucid-security universe deb http://security.ubuntu.com/ubuntu lucid-security multiverse clean http://archive.ubuntu.com/ubuntu
Once you are done with the configuration, start mirroring by issuing the below command
apt-mirror
Mirroring will now begin:
Downloading 146 index files using 20 threads... Begin time: Sat May 15 16:08:15 2010 [20]... [19]... [18]... [17]... [16]... [15]... [14]... [13]... [12]... [11]... [10]... [9]... [8]... [7]... [6]... [5]... [4]... [3]... [2]... [1]... [0]... End time: Sat May 15 16:08:22 2010 Proceed indexes: [SSPPPPPPPPPPP].....
It will show the total size of packages that is going to be downloaded.
Check whether the process is running properly
adminsage@adminsage-desktop:~$ ps aux | grep wget root 3267 0.0 0.0 5392 1704 pts/2 S+ 15:19 0:00 wget --limit-rate=100m -t 0 -r -N -l inf -o /media/var/spool/apt-mirror/var/archive-log.1 -i /media/var/spool/apt-mirror/var/archive-urls.1 root 3271 0.0 0.0 5392 1724 pts/2 S+ 15:19 0:00 wget --limit-rate=100m -t 0 -r -N -l inf -o /media/var/spool/apt-mirror/var/archive-log.5 -i /media/var/spool/apt-mirror/var/archive-urls.5 root 3272 0.0 0.0 5392 1720 pts/2 S+ 15:19 0:00 wget --limit-rate=100m -t 0 -r -N -l inf -o /media/var/spool/apt-mirror/var/archive-log.6 -i /media/var/spool/apt-mirror/var/archive-urls.6 root 3273 0.0 0.0 5392 1724 pts/2 S+ 15:19 0:00 wget --limit-rate=100m -t 0 -r -N -l inf -o /media/var/spool/apt-mirror/var/archive-log.7 -i /media/var/spool/apt-mirror/var/archive-urls.7 root 3277 0.0 0.0 5392 1700 pts/2 S+ 15:19 0:00 wget --limit-rate=100m -t 0 -r -N -l inf -o /media/var/spool/apt-mirror/var/archive-log.11 -i /media/var/spool/apt-mirror/var/archive-urls.11 root 3278 0.0 0.0 5392 1704 pts/2 S+ 15:19 0:00 wget --limit-rate=100m -t 0 -r -N -l inf -o /media/var/spool/apt-mirror/var/archive-log.12 -i /media/var/spool/apt-mirror/var/archive-urls.12 root 3279 0.0 0.0 5392 1724 pts/2 S+ 15:19 0:00 wget --limit-rate=100m -t 0 -r -N -l inf -o /media/var/spool/apt-mirror/var/archive-log.13 -i /media/var/spool/apt-mirror/var/archive-urls.13
We can see several threads of wget running simultaneously. Once the download is complete, you will see a message like below.
[20]... [19]... [18]... [17]... [16]... [15]... [14]... [13]... [12]... [11]... [10]... [9]... [8]... [7]... [6]... [5]... [4]... [3]... [2]... [1]... [0]... End time: Sat May 16 18:01:03 2010 20MB in 50092 files and 384 directories can be freed. Run /media/var/spool/apt-mirror/var/clean.sh for this purpose.
We are now done with the ‘mirroring’ of repository and all packages are now available on your machine ð Now we need to make this repo available to all other machines on the network via http. Install apache for this purpose.
sudo apt-get install apache2
Create Symbolic links
sudo ln -s /media/var/spool/apt-mirror/mirror/archive.ubuntu.com/ubuntu/ /var/www/ubuntu sudo ln -s /media/var/spool/apt-mirror/mirror/archive.canonical.com/ /var/www/canonical
Try accessing the repo locally from http://localhost/ubuntu and http://localhost/canonical
We have now successfully created the ‘reposerver’ and completed the stage1 of the whole process.
STAGE 2:
2) Setting up other machines/clients to use our server(reposerver) as the source repository .
NB’: The following steps must be done on the machine that we wish to update/upgrade using the ‘reposerver’, our local ububtu repository server.
Backup your current /etc/apt/sources.list
cp -av /etc/apt/sources.list /etc/apt/sources.list_backup
Modify sources.list to use our ‘reposerver’ as the repository. Consider the IP of ‘reposerver’ as 192.168.1.52
sudo gedit /etc/apt/sources.list
Replace the URLs http://archive.ubuntu.com/ubuntu/dists/lucid/Release with http://ip_of_reposerver/ubuntu/dists/lucid/Release and here it is 192.168.1.52
Use replace all option on gedit and replace like what is mentioned below.
Replace http://in.archive.ubuntu.com with http://192.168.1.52
http://archive.canonical.com with http://192.168.1.52
http://security.ubuntu.com with http://192.168.1.52.
The sample sources.list will be like below.
# deb cdrom:[Ubuntu 9.10 _lucid Koala_ - Release i386 (20091028.5)]/ lucid main restricted # See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to # newer versions of the distribution. deb http://192.168.1.52/ubuntu lucid main restricted deb-src http://192.168.1.52/ubuntu lucid main restricted ## Major bug fix updates produced after the final release of the ## distribution. deb http://192.168.1.52/ubuntu lucid-updates main restricted deb-src http://192.168.1.52/ubuntu lucid-updates main restricted ## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu ## team. Also, please note that software in universe WILL NOT receive any ## review or updates from the Ubuntu security team. deb http://192.168.1.52/ubuntu lucid universe deb-src http://192.168.1.52/ubuntu lucid universe deb http://192.168.1.52/ubuntu lucid-updates universe deb-src http://192.168.1.52/ubuntu lucid-updates universe ## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu ## team, and may not be under a free licence. Please satisfy yourself as to ## your rights to use the software. Also, please note that software in ## multiverse WILL NOT receive any review or updates from the Ubuntu ## security team. deb http://192.168.1.52/ubuntu lucid multiverse deb-src http://192.168.1.52/ubuntu lucid multiverse deb http://192.168.1.52/ubuntu lucid-updates multiverse deb-src http://192.168.1.52/ubuntu lucid-updates multiverse ## Uncomment the following two lines to add software from the 'backports' ## repository. ## N.B. software from this repository may not have been tested as ## extensively as that contained in the main release, although it includes ## newer versions of some applications which may provide useful features. ## Also, please note that software in backports WILL NOT receive any review ## or updates from the Ubuntu security team. deb http://192.168.1.52/ubuntu lucid-backports main restricted universe multiverse deb-src http://192.168.1.52/ubuntu lucid-backports main restricted universe multiverse ## Uncomment the following two lines to add software from Canonical's ## 'partner' repository. ## This software is not part of Ubuntu, but is offered by Canonical and the ## respective vendors as a service to Ubuntu users. deb http://192.168.1.52/canonical/ubuntu lucid partner deb-src http://192.168.1.52/canonical/ubuntu lucid partner deb http://192.168.1.52/ubuntu lucid-security main restricted deb-src http://192.168.1.52/ubuntu lucid-security main restricted deb http://192.168.1.52/ubuntu lucid-security universe deb-src http://192.168.1.52/ubuntu lucid-security universe deb http://192.168.1.52/ubuntu lucid-security multiverse deb http://192.168.1.52/ubuntu lucid main universe restricted multiverse deb-src http://192.168.1.52/ubuntu lucid-security multiverse
Update the repo by issuing the command
sudo apt-get update
You will not be able to install and update all packages from our ‘reposerver’.
But this isn’t enough if you wish to do a complete dist upgrade.The dist upgrade reads certain files and we need to modify these files to have an error free upgrade.
Dist-Upgradation:
Go back to our server (reposerver) and download the following packages under /var/www/
wget -c http://changelogs.ubuntu.com/meta-release wget -c http://changelogs.ubuntu.com/meta-release-lts
Create the directory:
sudo mkdir -p /media/var/spool/apt-mirror/mirror/archive.ubuntu.com/ubuntu/dists/lucid/main/dist-upgrader-all/current
change directory to /media/var/spool/apt-mirror/mirror/archive.ubuntu.com/ubuntu/dists/lucid/main/dist-upgrader-all/current
cd /media/var/spool/apt-mirror/mirror/archive.ubuntu.com/ubuntu/dists/lucid/main/dist-upgrader-all/current
And download the below packages under on the current directory, ‘/media/var/spool/apt-mirror/mirror/archive.ubuntu.com/ubuntu/dists/lucid/main/dist-upgrader-all/current’
wget -c http://archive.ubuntu.com/ubuntu/dists/lucid/Release wget -c http://archive.ubuntu.com/ubuntu/dists/lucid/main/dist-upgrader-all/current/ReleaseAnnouncement wget -c http://archive.ubuntu.com/ubuntu/dists/lucid/main/dist-upgrader-all/current/ReleaseAnnouncement wget -c http://archive.ubuntu.com/ubuntu/dists/lucid/main/dist-upgrader-all/current/lucid.tar.gz wget -c http://archive.ubuntu.com/ubuntu/dists/lucid/main/dist-upgrader-all/current/lucid.tar.gz.gpg
You are done with the server side modifications. Now go back to your client machine for which the distribution has to be upgraded. Modify the below files to point the URLs to our ‘reposerver’
sudo gedit /etc/update-manager/meta-release sudo gedit /usr/share/update-manager/mirrors.cfg
Sample files
/usr/share/update-manager/mirrors.cfg #ubuntu http://192.168.1.52/ubuntu/ http://192.168.1.52/ubuntu/ ftp://192.168.1.52/ubuntu/ ftp://192.168.1.52/ubuntu/ mirror://launchpad.net/ubuntu/+countrymirrors-archive http://ports.ubuntu.com/ ftp://ports.ubuntu.com/ http://ports.ubuntu.com/ubuntu-ports/ ftp://ports.ubuntu.com/ubuntu-ports/ http://old-releases.ubuntu.com/ ftp://old-releases.ubuntu.com/
modify the URL at /etc/update-manager/meta-release
# default location for the meta-release file [METARELEASE] URI = http://192.168.1.52/meta-release URI_LTS = http://192.168.1.52/meta-release-lts URI_UNSTABLE_POSTFIX = -development URI_PROPOSED_POSTFIX = -proposed
Update the repository
sudo apt-get update
To upgrade the distribution, issue the below command.
sudo do-release-upgrade
The machine will now automatically detects the new release and will ask confirmation for an upgrade. Voila! Everything is now local and we have saved huge bandwidth and ‘TIME’ ð
NB: Run apt-mirror at regular intervals using cron so that the packages will always be update and hence synced with the Ubuntu server.
