Tuesday, March 6, 2012

keeping packages locally

This might be useful if you have a limited internet band width. Just in case you need to reinstall or install from another machine, you do not need to download the packages again.

After the installation immediately before you run the update in your fresh installed suse run the command below, It will ask you the root's password and will tell you that rpm caching is enabled on the remote repositories.

Activate packages caching on your enabled repositories. Create a function like this and put it in /etc/bash.bashrc.local

keepackages () {
    if (( EUID !=0 )); then
        echo 'Root privileges are required for modifying system repositories.' 1>&2
        return 1
    fi
    while IFS="|" read -ra line; do
        [[ ${line[3]} = *Yes* ]] && zypper mr -k "${line[0]// /}"
    done < <(zypper lr)
}

Now you can just run   keepackages 


Backup the file  /etc/zypp/zypp.conf run the command:

  su root -c'cp /etc/zypp/zypp.conf  /etc/zypp/zypp.conf.orig'  

Edit that zypp.conf file run the command:

  printf '%s\n' 'g/^#[[:space:]]*download.use_deltarpm/s/^#[[:space:]]*/' w | ed -s /etc/zypp/zypp.conf   

 printf '%s\n' 'g/^\(download\.use_deltarpm[[:space:]]*=\)[[:space:]]*true$/s//\1 false/' w | ed  -s /etc/zypp/zypp.conf  

This section of that config file explains what it's doing. Of course you can just uncomment out that entry :-).

##
## Whether to consider using a .delta.rpm when downloading a package
##
## Valid values: boolean
## Default value: true
##
## Using a delta rpm will decrease the download size for package updates
## since it does not contain all files of the package but only the binary
## diff of changed ones. Recreating the rpm package on the local machine
## is an expensive operation (memory,CPU). If your network connection is
## not too slow, you benefit from disabling .delta.rpm.
##
# download.use_deltarpm = true

##
## Whether to consider using a deltarpm even when rpm is local
##
## Valid values: boolean
## Default value: false
##
## This option has no effect unless download.use_deltarpm is set true.
##
#  download.use_deltarpm.always = false

 

If you want to check which is option it is using you can run grep on it.

 grep -E -v '^($|#|;|\[)' /etc/zypp/zypp.conf 

You are now ready to update your system use the command below and accept the package key's.

 su root -c'zypper ref && zypper up' 

When it's done reboot just to be sure that update will take effect on your system and you can   find those rpm packages inside the directory  /var/cache/zypp/packages/   and it has it's own sub directories. Create a directory somewhere and copy those packages in there. While you are inside your home directory you can do the following commands.

Create a new directory.
   mkdir public_html/packages/  

Copy the rpm' from source to destination.
This rely on GNU/find and since suse use it by default it is not an issue and it should be used.
   find /var/cache/zypp* -name '*.rpm' -exec  cp -vu  {} public_html/packages/  \;

 New to bash 4.
 shopt -s globstar; rpm=(/var/cache/zypp*/**/*.rpm);  cp -vu "${rpm[@]}" public_html/packages 


This is a portable solution according to the demi god's with in the bash world! :-)
 find /var/cache/zypp* -name '*.rpm' -exec sh -c 'cp -vu -- "$@" public_html/packages/' _ {} + 

When you installed a package using zypper but it is a local install that rpm will be inside   /var/cache/zypper/  so using the   *   glob pattern matching will match zypper and zypp. So  you dont need to replace    /zypp   with  /zypper  to copy packages from those directory. Also it makes no sense to look for packages under   /var/cache   and as a normal user you don't have enough permission to poke into some directories under it.  That find command will search recursively for the file extension ending in rpm using also the (   glob ) and cp copies all of those files into your newly created directory, while the -vu flag means verbose and update so it only copies the new rpm from the source to it's destination and not copy everything again from the beggining.  You can change the path of the destination at your own heart's content but this place is important for the next step that will be describe about creating your local repository. You can run that command every time you install a new package. Just remember if you add a new repository before you install anything run the very first command in this guide so that package caching will take effect on that repo as well. When the time comes that rpm's are filling up your   /var/cache    directory you might need to delete them all, well you can just delete them at once using zypper. If you want to clear the   /zypper   directory you can use the find command as shown below. You can modify the path so you can delete other rpm's. We need to be root since that directories are restricted for read rights for normal users. Just remember to be careful when deleting files from the shell because it might not be recoverable.

Using zypper and it should be your first option.
  su root -c'zypper clean --all' 

 Search and Destroy rpm packages if zypper did not do it's job well!
   su root -c'find /var/cache/zypp*  -name '*.rpm' -delete' 

New to bash 4
 shopt -s globstar; rpm=(/var/cache/zypp*/**/*.rpm);  rm  "${rpm[@]}" 

Now what if you forgot to do rpm caching and did the update, is it still be possible to download the rpm and it's dependencies? The answer is yes, but only if those rpm packages are in your list of repositories. Fortunately zypper will just tell you if the packages that you are trying to re download is not found at any or your repos. Zypper will ask you to resolve some dependency issue and will give you some options, just don't choose the "break your system and be happy" option :-).  Thus we can run the command as root:


   readarray -t packages < <(rpm -qa --qf '%{NAME}\n'); zypper in -f -d "${packages[@]}"  

                                                         The other version :-)
  mapfile -t < <(rpm -qa --qf '%{NAME}\n'); zypper in -f -d "${MAPFILE[@]}"  

                              If you dont have bas4 which is highly unlikely but hey....
  IFS=$'\n' read -rd '' -a packages < <(rpm -qa --qf '%{NAME}\n'); zypper in -f -d "${packages[@]}"  

Just choose one but it is specific to bash 4 only. What it does is readarray aka mapfile load lines from a file or stream. The rpm command put it's output in that file or mapfile and then zypper read's those output one at a time. The -f switch is to do a force reinstall and the -d switch is download only,  just to download only and not install. To understand what's going on in that rpm command you can try to run it in a shell just remove it inside the open and close ( ) parenthesis.That odd looking "${packages[@]}" is indeed very useful since it quote all the element before zypper can read it and from experience if you have an rpm which name has a white space in between, it will save you the head ache :-). After it's done you can repeat  that find command so it will copy the newly downloaded rpm to your  public_html/packages    directory.

Now you have yourself a local copy of rpm in you and ready for future use. You can create a local repository using yast. Just point it to that directory (packages/) and your good to go. If you want to make a remote repository of those rpm files you can create a plain rpm repo using the package createrepo. You will need a running http server for that since it is the most easy to setup. The following steps will guide you to setup a local repository using the http protocol.

Install the pattern lamp_server and createrepo.

  su root -c'zypper in patterns-openSUSE-lamp_server createrepo' 


Go inside that packages/ directory and run createrepo.

 cd public_html/packages && createrepo .     #note the trailing white space and a dot.











 It will create a repodata/ directory, go inside that directory and

  cd repodata/ 

generate your key it will ask you some simple but important questions.

  gpg --gen-key 
























Check your generated key.

  gpg --list-key    












Create a  detached signature repomd.xml.asc to repo.xml.

  gpg -a --detach-sign repomd.xml 







Create a repomd.xml.key file and export the public key to it.

  gpg -a --export YOUR_PUBLICKEY > repomd.xml.key   #note the direction











Start your http serve.

  su root -c'rcapache2 start' 

Systemd from starting from 12.1
 systemctl restart apache2.service 












Now in your browser type the url below where username is the user in your system.

  localhost/~username  






















If that is successful  you can add that repo via zypper or yast. Use the command below to add that repo via zypper.

 su root -c'zypper ar http://localhost/~username/packages/ local' 













Where username is the name of the user and local will be the name of that repository, you can change that to whatever you want it to be called.

Refresh that newly added local repository.

   su root -c'zypper ref local' 

Accept the public key and your repo is good to go.








 



















If you are accessing it remotely meaning from another machine or from a virtual machine replace the localhost with it's correct ip which you can get from the out put of  ip r   Now you have your own remote repository in your  lan! If you added some new rpm packages on your repo you will need to run createrepo again so createrepo will know that there are some changes.


Enjoy!

The following links below should help you set up if not get an idea about setting up your repositories locally.

No comments:

Post a Comment