how to update all except packages with bugs

[superwow]

I have run apt-get update && apt-get upgrade several times in the past week or more and have had a number of list-bugs. All this occurred just after Debian promoted Jessie (not sure what the proper lingo for that is). Just after they did so, I got a large number of apps that needed updating, which struck me as strange since I run SID. Anyway, one of the packages threw a listbug, namely, unhide, required by rkhunter. Ok, so I waited not wanting to introduce a security issue. And in the past week, the number of apps needing update have grown. Now the number of apps needing update is huge, almost fills my terminal screen. And more bugs, all with security packages:

Code: [Select]

Fetched 6,278 kB in 11s (542 kB/s)                                                                             
Retrieving bug reports... Done                                                                                 
Parsing Found/Fixed information... Done                                                                       
serious bugs of libgnutls-deb0-28 (3.3.8-7 → 3.3.15-1) <Outstanding>                                           
 b1 - #784009 - [experimental] Lack of versioned symbols in nettle causes segfault                             
   Merged with: 784063                                                                                         
serious bugs of openssl (1.0.1k-3 → 1.0.2a-1) <Outstanding>                                                   
 b2 - #770605 - openssl: Removes symbol without SONAME bump                                                   
   Merged with: 768476 768522 769023 770278 771169 771993 781094 781929                                       
serious bugs of unhide (→ 20121229-1+b1) <Outstanding>                                                         
 b3 - #769345 - unhide: statically linked against libc6 without a Built-Using: field                           
Summary:                                                                                                       
 libgnutls-deb0-28(1 bug), openssl(1 bug), unhide(1 bug)   

So, I am wondering if there is a way to install every app that needs updating, but just not THOSE apps? (I have never pinned an app, is that the appropriate thing to do?)
Part two of that question would be how to have them properly update themselves whenever the bug goes away, without me having to do so manually?

[ozitraveller]

Yes there is

Code: [Select]

sudo apt-mark hold <package-name>

or

sudo apt-mark unhold <package-name>


Then just do the upgrade again.

I' m not aware of a way to do question 2.

Hope that helps

Ozi

[Snap]

That's the usual way but my crap memory constantly forgets what the heck I put on hold, specially when the bug fix takes a long time for a given package. I used to take notes of the packages in a text document... so I switched to pin the packages in /etc/apt/preferences.d/held. This way the packages are pinned to not update and keeping the version from testing, and get the reminder notes written at the same time anyway.

Code: [Select]

Package: packagename
Pin: release a=testing
Pin-Priority: 650
These are actually my notes on how I currently do it (I need to re-read them often): 

1st step - Check if the packages on hold are already fixed:

Login as su in a terminal

Code: [Select]

apt-listbugs list packagename
Release the held packages if they are fixed.

Case 1:   When no bugs are retrieved

Retrieving bug reports... Done
Parsing Found/Fixed information... Done

Code: [Select]

apt-get update && apt-get -d dist-upgrade
logout

logon as root,

Code: [Select]

init 3
apt-get update
apt-get dist-upgrade
If no bugs are retrieved, just keep going like this

Code: [Select]

apt-get clean
init 5 && reboot
If bugs are reported then proceed to the case 2 below.

Case 2:   When bugs are retrieved

If not in X better hit n, exit and login in an X session then run apt-get update && apt-get dist-upgrade and follow the next steps. If remaining outside X (in a tty) you can hit w and check the bugs in lynx or whatever CLI browser.

Code: [Select]

Retrieving bug reports... Done
Parsing Found/Fixed information... Done
grave bugs of apt (1.0.9.5 → 1.0.9.6) <Outstanding>
 b1 - #776910 - apt: upgrade from wheezy to jessie breaks in the middle
critical bugs of libwebkitgtk-3.0-0 (2.4.7-3 → 2.4.8-1) <Forwarded>
 b2 - #776686 - libwebkitgtk-3.0-0: Crash with SIGBUS in `WebCore::WidthIterator::advanceInternal`
Summary:
 apt(1 bug), libwebkitgtk-3.0-0(1 bug)
Are you sure you want to install/upgrade the above packages? [Y/n/?/...]
Then hit:

Code: [Select]

w
Check the bugs ( bookmark the pages) and study the cases.

In terminal hit:

Code: [Select]

n
in /etc/apt/preferences.d/held-packages file pin the packages to hold

Code: [Select]

Package: packagename
Pin: release a=testing
Pin-Priority: 650
Save & close. Back to terminal:

Code: [Select]

aptitude update && aptitude -s dist-upgrade
Check the proposed solutions. Choose the one to keep the installed packages and remember it to use it later.

Code: [Select]

aptitude update && aptitude -d dist-upgrade
Choose the right option, let it run and logout.

logon as root,

Code: [Select]

init 3
aptitude update
aptitude dist-upgrade
aptitude clean
init 5 && reboot
Hope this helps.

[VastOne]

I have run apt-get update && apt-get upgrade several times in the past week or more and have had a number of list-bugs. All this occurred just after Debian promoted Jessie (not sure what the proper lingo for that is). Just after they did so, I got a large number of apps that needed updating, which struck me as strange since I run SID.

This is very typical with a Debian freeze and then the subsequent Debian release... When the freeze hits testing, there is no place any more to progress applications through the normal processes. Literally no place to park apps as they update.  A backlog begins and once it is lifted and the proper testing channels flow begins again, you inherently see all the apps that have been sitting on hold waiting for the process to begin again explode from the gates

This happens every time with a Debian release and one to expect... It is a frustrating process for SID users, but an absolutely needed one

[superwow]

Thanks all for the info. I knew there was a way to do this. Ozi, thanks for the reminder. I think I have used this in the past and ... <clears throat> ...  forgot. This was what I was hoping someone would tell me. If I hold packages this way, is there a way to check at some point which packages have been held? I am assuming that data is  stored in /etc/apt/preferences.d? I found that I can do "sudo apt-mark showhold" but it does not show all the apps I have held. Interesting. No matter how many times I try to hold "unhide" it will not hold, and will not show as being held when invoking "sudo apt-mark showhold".

Snap, dang that is more complicated than I figured. I am theoretically aware of the init levels but why 3 and 5 specifically? <shows newb status> Thanks for sharing the way you manage upgrades. Very technical.

V-ger, that's what I have heard. I just haven't been on a Debian distro when that happened. Last upgrade of the same sort I was using our largest cousin distro, when things broke horribly all the time, arg. But having gone through the systemd update recently, I have come to appreciate not accepting all upgrades I can handle the massive update though. No big deal compared with

[Snap]

To stay away from the X server during upgrades. machinebeacon from the BBQ explains it better than me:

- If possible, do this outside of X (init 1) Use init 3.
- Read the fscking output. Read what is going to be removed. If there are more packages removed than downloaded or upgraded, be careful. Sid receives updates four times a day. Waiting is always a good choice, use the time to visit the forums and see if there are upgrade warnings.

- Be careful if there are transitions happening in sid (bigger version changes of big packages, like perl, gcc) - they need a few days to be fully available in the repositories, and a typical feature is a huge number of removed packages. In this case do not upgrade. Wait a few days, usually after 2-3 days the transition is completed (for the common architectures, at least)
-machinebacon.

This is another interesting reading from the BBQ for those dealing with Sid

From Debian & Co. to BBQ

“Maybe you are running some kind of Debian (derivative) and want to have a grill base. Not that we encourage users to join the BBQ via destroying their perfectly working Debian stable or testing system - but I have recently got a PM in which I was asked a few questions, and I make the answer public, for everybody to see, yesh."-machinebacon
Q: I use <Debian distro of your choice> and want to get the BBQ, how should I proceed?
A: We need to edit the sources.list and the APT preferences.
/etc/apt/sources.list.d/bbq.list

## bbq
deb http://linuxbbq.org/repos/apt/debian sid main

## debian
deb http://http.debian.net/debian/ unstable main contrib non-free

## debian experimental - don't shit your pants
deb http://http.debian.net/debian/ experimental main
#deb-src http://http.debian.net/debian/ unstable main

## siduction base (stuff like the towo kernels or deadbeef comes from here, just sayin')
deb http://packages.siduction.org/base unstable main contrib non-free
#deb-src http://packages.siduction.org/base unstable main contrib non-free

## extras (you can have this commented)
#deb http://packages.siduction.org/extra unstable main contrib non-free
#deb-src http://packages.siduction.org/extra unstable main contrib non-free 

## user repo (you can have this commented)
#deb http://packages.siduction.org/user unstable main contrib non-free
#deb-src http://packages.siduction.org/user unstable main contrib non-free

## siduction's fixes, please don't change (they saved many asses)
deb http://packages.siduction.org/fixes unstable main contrib non-free
#deb-src http://packages.siduction.org/fixes unstable main contrib non-free

## experimental from siduction (again, don't be a pansy)
deb http://packages.siduction.org/experimental unstable main contrib non-free
#deb-src http://packages.siduction.org/experimental unstable main contrib non-free
/etc/apt/preferences.d/10siduction
Package: *
Pin: release l=SiductionExperimental
Pin-Priority: 100
/etc/apt/apt-file.conf
# Apt-file configuration file

# Substitutions are made as follows:
#   host => remote hostname
#   port => remote port
#   uri => complete URI from sources.list
#   path => path from /
#   dist => the distribution name
#   cache => path to the local cache dir
#   dest => the destination file name inside the cache dir
#   cdrom => cdrom mount point

# Where are located Packages
destination = <host>_<path>_dists_<dist>_Contents-<arch>.gz

# common code blocks can be defined as variables and be used as $check_cmd, etc. later
check_cmd = ( ( gunzip -l "<cache>/<dest>_tmp" >/dev/null 2>&1 || (echo "File is not gzipped.\
"; false) ) && mv "<cache>/<dest>_tmp" "<cache>/<dest>" 2>&1 )
error_cmd = ( rm -f "<cache>/<dest>_tmp"; echo "Can't get <uri>/dists/<dist>/Contents-<arch>.gz" )
post_dl_cmd = $check_cmd || $error_cmd


# Fetch methods using diffindex-download:
# -i : ignore missing files
# -q : be quiet
# -n <num> : download full file if more than <num> patches would be necessary
http = diffindex-download -i "<uri>/dists/<dist>/Contents-<arch>.gz" <cache>/<dest>
https = diffindex-download -i "<uri>/dists/<dist>/Contents-<arch>.gz" <cache>/<dest>
ftp = diffindex-download -i "<uri>/dists/<dist>/Contents-<arch>.gz" <cache>/<dest>
# In debtorrent URLs, we have to replace 'debtorrent' by 'http', and we always download the full file
debtorrent = diffindex-download -i -n 0 "http://<host>:<port|9988><path>/dists/<dist>/Contents-<arch>.gz\
" <cache>/<dest>

ssh = scp -P <port|22> "<user>@<host>:/<path>/dists/<dist>/Contents-<arch>.gz" "<cache>/<dest>_tmp" && $post_dl_cmd
rsh = rcp -l <user> "<host>:/<path>/dists/<dist>/Contents-<arch>.gz" "<cache>/<dest>_tmp" && $post_dl_cmd
file = cp "/<path>/dists/<dist>/Contents-<arch>.gz" "<cache>/<dest>"
copy = cp "/<path>/dists/<dist>/Contents-<arch>.gz" "<cache>/<dest>"
cdrom = echo "Put CDROM labeled <path> in the cdrom device and press [ENTER] \
" > /dev/stderr ; read DUMMY ; mount "<cdrom>"; cp "<cdrom>/dists/<dist>/Contents-<arch>.gz\
" "<cache>/<dest>" ; umount "<cdrom>"

# Schemes that might require user input on 'apt-file update'
# These will be skipped if -N is given
interactive = cdrom rsh ssh
/etc/apt/apt.conf.d/80siduction
// apt defaults for siduction
// apt 0.7 introduces automatic behaviour unsuitable for sid, revert this

// auto-remove breaks on meta packages
APT::Get::AutomaticRemove "0";
APT::Get::HideAutoRemove "1";

// Recommends are as of now still abused in many packages
APT::Install-Recommends "0";
APT::Install-Suggests "0";
Debug::pkgAutoRemove "0";

// PDiffs reduce the required download for apt-get update, but increase the
// CPU requirements and quite often fail.
// Acquire::PDiffs "0";
/etc/apt/apt.conf.d/01autoremove APT {
 NeverAutoRemove
 {
  "^firmware-linux.*";
  "^linux-firmware$";
  "^linux-image.*";
  "^kfreebsd-image.*";
  "^linux-restricted-modules.*";
  "^linux-ubuntu-modules-.*";
  "^gnumach$";
  "^gnumach-image.*";
 };
 Never-MarkAuto-Sections
 {
  "metapackages";
  "restricted/metapackages";
  "universe/metapackages";
  "multiverse/metapackages";
  "oldlibs";
  "restricted/oldlibs";
  "universe/oldlibs";
  "multiverse/oldlibs";
 };
};
/etc/apt/apt.conf.d/10apt-listbugs (install apt-listbugs, it will create this file)
// Check all packages whether they has critical bugs before they are installed.
// If you don't like it, comment it out.
//DPkg::Pre-Install-Pkgs {"/usr/sbin/apt-listbugs apt || exit 10";};
//DPkg::Tools::Options::/usr/sbin/apt-listbugs "";
//DPkg::Tools::Options::/usr/sbin/apt-listbugs::Version "2";
// AptListbugs::IgnoreRegexp "FTBFS";
/etc/apt/apt.conf.d/70debconf
// Pre-configure all packages with debconf before they are installed.
// If you don't like it, comment it out.
DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt || true";};
/etc/apt/apt.conf.d/50unattended-upgrades
// Unattended-Upgrade::Origins-Pattern controls which packages are
// upgraded.
//
// Lines below have the format format is "keyword=value,...".  A
// package will be upgraded only if the values in its metadata match
// all the supplied keywords in a line.  (In other words, omitted
// keywords are wild cards.) The keywords originate from the Release
// file, but several aliases are accepted.  The accepted keywords are:
//   a,archive,suite (eg, "stable")
//   c,component     (eg, "main", "crontrib", "non-free")
//   l,label         (eg, "Debian", "Debian-Security")
//   o,origin        (eg, "Debian", "Unofficial Multimedia Packages")
//   n,codename      (eg, "jessie", "jessie-updates")
//     site          (eg, "http.debian.net")
// The available values on the system are printed by the command
// "apt-cache policy", and can be debugged by running
// "unattended-upgrades -d" and looking at the log file.
//
// Within lines unattended-upgrades allows 2 macros whose values are
// derived from /etc/debian_version:
//   ${distro_id}            Installed origin.
//   ${distro_codename}      Installed codename (eg, "jessie")
Unattended-Upgrade::Origins-Pattern {
        // Codename based matching:
        // This will follow the migration of a release through different
        // archives (e.g. from testing to stable and later oldstable).
//      "o=Debian,n=jessie";
//      "o=Debian,n=jessie-updates";
//      "o=Debian,n=jessie-proposed-updates";
//      "o=Debian,n=jessie,l=Debian-Security";

        // Archive or Suite based matching:
        // Note that this will silently match a different release after
        // migration to the specified archive (e.g. testing becomes the
        // new stable).
//      "o=Debian,a=stable";
//      "o=Debian,a=stable-updates";
//      "o=Debian,a=proposed-updates";
        "origin=Debian,archive=stable,label=Debian-Security";
};

// List of packages to not update
Unattended-Upgrade::Package-Blacklist {
//   "vim";
//   "libc6";
//   "libc6-dev";
//   "libc6-i686";
};

// This option allows you to control if on a unclean dpkg exit
// unattended-upgrades will automatically run
//   dpkg --force-confold --configure -a
// The default is true, to ensure updates keep getting installed
//Unattended-Upgrade::AutoFixInterruptedDpkg "false";

// Split the upgrade into the smallest possible chunks so that
// they can be interrupted with SIGUSR1. This makes the upgrade
// a bit slower but it has the benefit that shutdown while a upgrade
// is running is possible (with a small delay)
//Unattended-Upgrade::MinimalSteps "true";

// Install all unattended-upgrades when the machine is shuting down
// instead of doing it in the background while the machine is running
// This will (obviously) make shutdown slower
//Unattended-Upgrade::InstallOnShutdown "true";

// Send email to this address for problems or packages upgrades
// If empty or unset then no email is sent, make sure that you
// have a working mail setup on your system. A package that provides
// 'mailx' must be installed. E.g. "user@example.com"
//Unattended-Upgrade::Mail "root";

// Set this value to "true" to get emails only on errors. Default
// is to always send a mail if Unattended-Upgrade::Mail is set
//Unattended-Upgrade::MailOnlyOnError "true";

// Do automatic removal of new unused dependencies after the upgrade
// (equivalent to apt-get autoremove)
//Unattended-Upgrade::Remove-Unused-Dependencies "false";

// Automatically reboot *WITHOUT CONFIRMATION* if a
// the file /var/run/reboot-required is found after the upgrade
//Unattended-Upgrade::Automatic-Reboot "false";


// Use apt bandwidth limit feature, this example limits the download
// speed to 70kb/sec
//Acquire::http::Dl-Limit "70";

Extra Q & A

Make sure you have no duplicates in the sources.list.d files. If you come from a distro that contains gtk3-applications, you are advised to remove them before performing an update && dist-upgrade. The BBQ takes no responsibility if you burn your meat to a crisp.
Q: I don't run a siduction kernel, can I still fetch packages from the siduction or BBQ repos?
A: Definitely. You can use any kernel you like (preferably 3.2.0-3 and higher) with a mix of repositories.
Q: I see you have experimental in the sources.list - that might break my system?!
A: Not by default. Usually all packages are sucked from the standard Debian sid sources, and if there are fixes at siduction, they are gotten from there. If you want to install packages from experimental, you have to do this manually and tell APT that you want to install from a certain source. I will *not* give the instructions here to prevent borkage by randomness. A little tip: in APT you can add the -t option to choose a certain repository to download from. Also, there's the APT preferences file.
Q: Can I downgrade packages in Sid?
A: Technically yes, practically it might cause some buttache. Single packages without dependencies are easy to downgrade. If there is a chain of dependencies behind it (for example a certain downgraded package depends on an earlier version of gcc, which in turn depends on an earlier version of binutils, you might run into an PITA situation) -- if you want to sleep with your wife, you don't hang a banana around your hips, do you? Same about Unstable - either you go for it, or not. There's no way to be "a bit pregnant" or "a bit dead".
Q: I need even newer package versions than in Sid or experimental. Add <packagename> to the BBQ!!11
A: You need that, huh? Then go to github or sourceforge, grab yourself the source and happily compile. The BBQ staff is not working on your cotton field.
Q: Can I run a mix of kernels?
A: Yes. You can install as many kernels as there's space for on your drives. They will be listed in the "Extended" menu of GRUB. smxi can do kernel installations for you, sane choices are Debian's stock kernel, the Debian experimental kernels, siduction's towo kernel, the slh Aptosid kernel, the Liquorix kernel and the BBQ realtime kernel.

[VastOne]

In all my dealings with Debian SID, it has only been a matter of understanding the output of apt-get update && apt-get dist-upgrade

In all my dealings with Debian SID, the only heartburn I have ever had in 7+ years was Perl updates.. The perl devs were eviscerated for how both of those updates caused so much chaos and I doubt we will ever see that again

Siduction has morphed into it's own version of SID and is far and away different than what we deal with here with VSIDO

KISS... nothing more, nothing less ... the goal here is to make SID easy, presentable and feasible... I think we have scored on all of those goals

[PackRat]

I am theoretically aware of the init levels but why 3 and 5 specifically

Run levels 2-5 are the multi-user runlevels; the default runlevel (typically 2 for Debian systems) is run at boot time and typically starts the X server so you can log into a graphical environment. Usually, one of the remaining multi-user run levels does not start the X server. If you log into your tty as root (or use sudo) and then switch to the non X runlevel with - for example:

Code: [Select]

init 5
you will switch your runlevel and shut down the X server. A lot of people recommend that dist-upgrades be run after the X server is shut down - especially if Xorg is getting a bunch of updates.

init 3 and init 5 are the other runlevels that are traditionally used - although, I'm pretty sure Slackware boots runlevel 4 by default.

Runlevel 0 is run at shoutdown, runlevel 1 is single user mode (used for the repair mode options in the grub menu), runlevel 6 is for rebooting.

 

[PackRat]

In all my dealings with Debian SID, the only heartburn I have ever had in 7+ years was Perl updates..

I remember that. The only other issue i've had was with gnome libraries lagging behind the rest of gnome. All the gnome apps went down for about a week until the rest of the upgrades came through. So it's worth repeating:

it has only been a matter of understanding the output of apt-get update && apt-get dist-upgrade

[Snap]

In all my dealings with Debian SID, the only heartburn I have ever had in 7+ years was Perl updates.. The perl devs were eviscerated for how both of those updates caused so much chaos and I doubt we will ever see that again

I've heard of the perl nightmare but never faced it myself. Maybe I'm too new here.

Siduction has morphed into it's own version of SID and is far and away different than what we deal with here with VSIDO

Yep, Siduction is damn too complex. That's why I always stepped back the many times I've tried it. Vsido makes the Sid side of life easier.

In all my dealings with Debian SID, it has only been a matter of understanding the output of apt-get update && apt-get dist-upgrade

It seems I'm going the complex route, isn't it? Maybe because I've learned the Sid business with Siduction and some BBQ flavors prior to discover Vsido. So any guidance to simplify the process is very welcome. Now I have to figure out what your sentence implies regarding the outputs. 

@ PackRat
Thanks for the insight about the run levels. Never found a simpler explanation of them.

[superwow]

All, thanks for the advice.

@Snap, I have seen that post/thread by machinebacon before - a wealth of info that guy, about linux and tobacco. Just forgot to search the q before posting here. And quite a sources.list.

@Packrat, thanks for the info about runlevels. More detailed yet simpler than the man pages. Thanks.

So,  I followed the advice to hold the package. Unhide will not hold. So I held rkhunter.

Now to show held packages I

Code: [Select]

apt-mark showhold

and to list the bugs of each package which I have held, or for any package for that matter, I can

Code: [Select]

apt-listbugs list openssl

and get the information I want.

I guess I could write a script to index the list of held apps and pass that list to apt-listbugs and check to see if there is output and if not remove the hold. But my shell coding skills need to increase prior to that

Thanks for the info everyone.

[ozitraveller]

@ALL - a fascinating read guys!!! As of a couple of weeks ago I stopped doing a dist-upgrade in favour if just upgrade. Doing a dist-upgrade at this point would wipe out more than half of my system.

I'm sure you have all read this, but I'll add it for completeness anyway.
What are some best practices for testing/sid users?

Ozi