Upgrade Mandriva 2010 to 2011

Part of the fun of Linux, for me at least, is the ability to fix things when they go wrong. And things generally go wrong when you don't follow the advice of conventional wisdom. Plus, my personal mantra is "fix it until it breaks".

One of those great pieces of common wisdom when discussing Mandriva is that you only upgrade within major versions, like from Mandriva 2010.1 to 2010.2, but when moving between major versions, like from Mandriva 2010 to 2011, you do a clean install.

If you have followed other convention wisdom, you have a separate physical partition for /home. With that configuration in place, you can do a clean install of the root filesystem and keep all you personal files and settings. But that can also pose problems since there is not always a clean upgrade path for the personal configuration files that live in the dotfile and hidden directories of your home directory. More on that at another time.

I wanted to upgrade (not update) from Mandriva 2010.2 to 2011.0 just to see what would happen. A look at the Notes for 2011.0 would indicate that such an option is possible with relative ease. Well, not really. Here's what you need to do.

After you have made certain that all you data is backed up and secure, the first step of the update is to update all the packages on your system.

# urpmi --auto-update --auto -v

Next, all the references to the 2010.2 repositories must be deleted.

# urpmi.removemedia -av

Then, 2011.0 repositories must be added. I also use the PLF repositories, so those need to be added those as well. You can do this automatically with EasyURPMI at PLF.

For the full Mandriva repos (all on one line):

# urpmi.addmedia --distrib --mirrorlist 'http://api.mandriva.com/mirrors/basic.2011.0.$ARCH.list'

For the PLF repos (all on one line):
# urpmi.addmedia --distrib --mirrorlist 'http://plf.zarb.org/mirrors/2011.0.$ARCH.list'

Note that the commands contain the variable $ARCH. I did that so you could copy the command and it would automatically detect your i586 or x86_64 system as appropriate.

Even after this has been done, some of the individual repos (the non-free ones) that are deliberately not enabled. To enable these, in your menu go to Tools > System Tools > Configure Your Computer. This launches the Mandriva Control Center. Then select Software Management > Configure media sources. Tick the boxes next to the repos Main Backports, Contrib Backports, Non-free, Non-free Updates, Non-free Backports, PLF Non-free and PLF Non-free Backports. Click on OK when you are all done and exit the Control Center.

If you want to avoid the lengthy trip through the menu system, execute drakrpm-edit-media in a terminal window.

If anyone is aware of a script that can have the same result, I would like to hear of it.

I have always found it better to do this kind of upgrade without the GUI running. So far, if you've been following along on your own system preparing for the upgrade, you've been able to cut and paste commands between your browser and a terminal window, but once the GUI disappears, that becomes impossible. Since the command to perform the actual update is a long one, let's create a shell script to execute it.

Using whatever text editor you prefer, create a file named 2011_upgrade.sh, make it executable with chmod +x and have it contain the following (all on one line):

#!/bin/sh urpmi --auto --auto-update --replacefiles 2>&1 | tee 2011_upgrade.log # upgrade 2010.2 to 2011.0

Now, using Crtl-Alt-F2, open a virtual terminal and log in as root. Stop the dm service to shut down the window manager with:

# service dm stop

Now log back in as root, change to your home directory and run the 2011_upgrade.sh script.

The urpmi package manager now wants to install a lot of packages and urpmi is smart enough to know to install certain system-critical packages first. But urpmi cannot install python-numpy because if version conflicts. This causes the upgrade process to fail to install any python packages which will seriously munge your system and which prevents the installation of packages which depend n the newer version of python..

Don't panic and most importantly, don't reboot. Your system is running now and there is no guarantees that it will on a reboot.

The solution is simple. That is we should uninstall python-numpy manually and then re-install the most current version; this will allow urpmi to update python and unblock the installation of many more packages.

The problem we encounter is that the urpme command doesn't know how to uninstall python-numpy without also uninstalling every other package that depends on it. Normally that's a good behavior, but not for in this instance.

We'll need to use the rpm command to uninstall only the one package and then use it's smarter cousin urpmi to successfully update all of python.

# rpm -ev --nodeps python-numpy

and then,

# urpmi python-numpy

There may be similar notifications for other files. I was notified about libkexiv2 and x11-driver-input-evtouch. The same trick using rpm -ev --nodeps works with them as well.

With these problems fixed, run the 2011_update.sh script once more. Keep fixing and re-running until there are no more packages to update. Once that state is reached, it will be safe to reboot into Mandriva 2011.0. When 2011.1 is released, the same technique can be used, but there should be fewer problems.

One problem that did occur post-upgrade concerns networking. Mandriva 2011 implements a new network manager and your old network configuration files won't work with it. While you can use the network section of the Mandriva Control Center to fix this (I had to delete the connection and re-create it to make it work), it's easier to add NM_CONTROLLED=yes to the end of each ifcfg-* file in /etc/sysconfig/network-scripts. That's all that is necessary for the new Network Manager to control them.

Otherwise, everything works fine and I'm now running Mandriva 2011.0. That the upgrade worked as smoothly as it did is a testament to the work and expertise of the Mandriva team. If I come across other issues, I'll address them here.

More fun with Linux

There are two things that can help you to have some fun with Linux.

The first is VirtualBox. Since you are not using it for commercial purposes, you may download and install the full-featured version for your host operating system. If you are a MS Windows user curious about Linux, this is ideal for you. If you are a Linux user curious about different distros, this works as well.

After you install VirtualBox on your host operating system, you need to learn how to use it. the VirtualBox site has excellent documentation. My suggestion for configuration are give the virtual machine the maximum amnount of RAM and video memory that is practical and set the network choice to "bridged" rather than "NAT".

Next stop is to download the NetbootCD. This CD contaions enough of an OS to enable wired Ethernet connectivity and download and install several Linux distros via FTP.

What can you install?

Ubuntu

Debian GNU/Linux

Fedora

openSUSE

Mandriva Linux

Scientific Linux

CentOS

Slackware

Now you can create a virtual machine for each distro and experiment to your heart's content. This is much safer and easier to implement that dual- or multi-booting.

Remember that when using VirtualBox, you are only exposing the virtual hardware to Linux, not the real hardware of your machine, so learn how to identify your real hardware and learn how to use Google to determine the level of Linux support for it.

One last suggestion. When you do decide to install a Linux distro on your machine "for real", save yourself some headaches and obtain a second hard drive for your Linux install. If your hardware permits booting from an external USB drive and the distro you have chosen permits installing on a USB drive, you are home free. Otherwise, you'll need to install the second drive as a temporary replacement to get Linux installed. Then you can move it to an external USB enclosure .

If you BIOS does not permit booting from an external USB drive, you might find plpbt4win to be useful. It is a special bootloader that can boot an OS from and external USB drive. There is a Linux version as well.

Have fun!

Mageia

The first release of Mageia 1 is out. You can download it from here and read the release notes here.

If you've used Mandriva Linux, you'll feel right at home with Mageia. Mandriva has always had a reputation of working well for both the novice user and the power user. Installation is typically easy and hardware support is among the best of any distro. The user and administrative tools are comprehensive and easy to use. Of course, under the hood, it's all Linux and all configuration files are plain text files and all a competent administrator needs do is to choose a shell and a text editor.

Mandriva has also been known for excellent default fully-featured configurations of the KDE, GNOME, XFCE and LXDE desktop environments as well as a broad range of available applications. Mageia is no exception in thsi regard.

For the average user, the changes are mostly cosmetic. The Mageia art, colors and graphics are very well done (and contributed by the Mageia user community). But if it's just a pretty Mandriva, why is there a need for another Linux distro?

Mandriva began life as Mandrake Linux, based on Red Hat 5; Mandrake was essentially "Red Hat Done Better". It offered a well-configured KDE as the default desktop as well as its own administrative tools that provided more flexibility than those tools offered by Red Hat. The under-the-hood difference in these tools was that they were (and continue to be) written in perl language rather than the python language as used by Red Hat. Mandrake/Mandriva also spent considerable effort on its unique urpmi package management tool, essentially an intelligent, powerful console/gui wrapper around the RPM system of package management.

At some point, the business management of Mandrake/Mandriva lost its way, rallied, re-grouped and then lost its way again. The first time, they lost focus on Linux and the second time, they lost focus on their community. Mageia stepped in to fill the void in the community so that all the good things that were represented by the Mandriva distro would not be lost in the corporate buyout.

Which is why the software is so similar and the community is so different; you only need to fix what is broken.

Mageia offers the most recent verson of your favorite application software and all the major software you expect to be there is there. Mageia uses kernel 2.6.38.7 with additional patches for automatic process grouping andTransparent Huge Page support as well as support for AMD Fusion APUs and Intel Sandy Bridge and Panther Point chipsets.

Both 32-bit and 64-bit installation DVDs as well as a Live CD are available. Migration from existing Manriva 2010.2 installations is easily done. A large on-line software repository is available and provides a wide range of software of both free licensed and other licensed categories (the latter is not enabled by default, but is opt-in).

As with any initial release, you're likely to find a few minor bugs, but the Mageia community is there to help you. As a long-time user of Mandrake/Mandriva and co-author with Bill Ball of the Red Hat/Fedora Unleashed series, I believe you'll find Mageia a full-featured and easy-to-use Linux distro.

A Faster System

I'm always looking for ways to speed up my computer, especially when it comes to the boot process and loading applications. A good overall discussion or improving system performance is at https://wiki.archlinux.org/index.php/Maximizing_Performance, just remember that it focuses on the ArchLinux distro, not the Mandriva distro I use.


READAHEAD
A useful utility is readahead. According to its manpage, readahead populates the page cache with data from a file so that subsequent reads from that file will not block on disk I/O. On my Mandriva 2101.2 system, the default installation of readahead places a default file at /var/lib/readahead/default.early. To create custom files based on your system, you should reboot the system and pause the GRUB menu and add the following to the command line.

init=/sbin/readahead-collector

This will examine your system as it boots and create a list of what files are to be loaded in the order that they are loaded.

The configuration file is found at /etc/readahead.conf and this is where you can turn on debugging if you're experiencing problems and can add excluded directories.

PRELINK
Next is the prelink utility. According to the man page, prelink is a program that modifies ELF shared libraries and ELF dynamically linked binaries in such a way that the time needed for the dynamic linker to perform relocations at startup significantly decreases. Due to fewer relocations, the run-time memory consumption decreases as well (especially the number of unsharable pages).

The file /etc/prelink.conf contains a list of files and directories that prelink uses to determine where it should look and what files to avoid. Prepending -l (or -h if on a networked filesystem) to a path makes prelink search that directory for ELF binaries or shared libraries. You can prepend -b to blacklist a directory or a file to avoid which will speed up the search or deliberately avoid prelinking certain binaries.

Mandriva provides a directory to keep your customizations separate at /etc/prelink.conf.d/; just create a file there ending in .conf and your customizations will be included. The general invocation is

# prelink -a

SPEEDBOOT
Finally, on Mandriva (through 2010), speedboot is installed and you can examine the configuration file at /etc/sysconfig/speedboot. By default, the value os set to auto. You enable speedboot by temporarily adding speedboot=probe to the GRUB command line as you did for readahead-collector above. Other distros may use fastboot.

Reverse SSH Tunneling

I have a small auto shop and having Internet access there comes in handy when I need to find auto-related information, order parts or just surf the web and do some email. Until recently, I've used a EVDO modem and got 10x dial-up speeds, but at $65 a month, it just got to be too great an expense for such low speeds. My next-door neighbor came to the rescue and allowed me to share his fiber-optic Internet access via a wireless connection.

But that arrangement came with a downside. Previously, I had left the EVDO connection open and could ssh into my work computer (all my systems run Linux). But now I had no access to the router/firewall and when I asked my friendly neighbor about port-forwarding an ssh connection, all I got was a blank stare.

Luckily, Jason from my TWUUG group suggested tunneling over ssh, diagrammed in the image above. This is how it's done.

On my shop computer, I run:

$ ssh -R 2222:localhost:22 hoyt@home.ip.net

and then on my home computer, I run:

$ ssh -p 2222 hoyt@localhost

And I have a console window opened to my shop machine.

Jason also suggested "You may want to look at '-f' or '-N' in ssh(1) depending on if you are using certs or not. Take a look at ssh_config(5) for setting up your ~/.ssh/config on a per host basis. This way, you can auto-setup port forwarding, odd port numbers, etc." In other words, it is possible to have the command execute automatically every time I start or re-start the machine at the shop and stay in the background.

The -f tells ssh to go into the background just before it executes the command.

The -N instructs ssh to not execute a command on the remote system.

REFERENCES
Quick-Tip: SSH Tunneling Made Easy
A few examples including using this technique to tunnel past a firewall to use Jabber.

Breaking Firewalls with OpenSSH and PuTTY
A Windows-oriented tutorial.

SSH Port Forwarding (SSH Tunneling)
Additional examples and an ASCII diagram of how it works.

How to make your web browser think it is at Fermilab
Use ssh tunneling to use a remote machine as a proxy.

Reverse SSH Tunneling, Bypassing Firewalls and NAT
using an intermediate machine to establish a reverse ssh tunnel.

Google is your friend.

Running msec on on of my Mandriva machines turned up a number of world-writable files. These are potential security problems, especially on a machine that faces the Internet. The obvious solution was to use chmod to remove any world writable permissions.

But there's the problem. The chmod command has a recursive directive, -R, that would work except that it would change permissions on the directories as well. After all in *NIX, all directories are files.

Thanks to my friends at TWUUG, I discovered that the find command, used with its -exec directive could accomplish what I wanted.

find /xxx -type f -exec chmod o=-w {} + ;

Breaking this down:

find - the find command itself. Depending on what files you are modifying, you may need to be root.

/xxx - the path to where the files are located.

-type f - this tells find to look for files; type d would tell it to look for directories.

-exec - this directive tells find to execute the command that follows, in this case:

chmod - the command that will change permissions.

o=-w - the directive to chmod that sets the permissions of others to not writable. You can use whatever incantations of permissions that you need (see man chmod).

{} + - this appends each file selected by the find command to those that chmod will act upon.

`\' - Note: these characters are used to protect the expansion started by {} from the shell. I found that not only was it unnecessary on my Mandriva system, including it would cause the command to fail. YMMV.

; - this terminates the command.

Going Postal with Postal2

I have always enjoyed computer games that are out of the ordinary. Postal2 by Running With Scissors is certainly that. It is both amusing and offensive in a very politically incorrect way, featuring Gary Coleman, the Taliban and dead cow heads.

At postal2.maximumhoyt.com, I am in the process of documenting how to install and run the game on a Linux computer as well as run the win32-only 1409X version of the dedicated Postal2 server. The site is also providing me with experience in site development using SSI (Server Side Includes) and Cascading Style Sheets (CSS).