VSIDO Community

VSIDO Support => Scripts and How To's => How To's => Topic started by: VastOne on February 03, 2016, 08:50:03 PM

Title: How To: Building the VSIDO Installer
Post by: VastOne on February 03, 2016, 08:50:03 PM
For several years I have looked at different Linux installers and what could be done in the debian installer space.  I have always come away disappointed that what was/is out there was seldom documented or shared in a way that could be used anywhere else

Two weeks ago misko_2083 introduced us to Trios Linux (http://vsido.org/index.php?topic=1104.msg12566#msg12566).  For the first time I got to see an  installer I was interested in (semplice linstaller) (https://github.com/semplice/linstaller) used in a practical debian setup I was familiar with

Since then there has been non stop development and collaboration with Filip Danilović and Miloš Pavlović that started here (http://vsido.org/index.php?topic=1124.msg12586#msg12586) and was discussed more here (http://vsido.org/index.php?topic=1129.msg12664#msg12664).  The end result is VSIDO now has a new installer that will continue to grow

For complete transparency these are the steps I use to convert the TRIOS installer to the VSIDO installer.  This is for anyone in the future looking for these steps, who may find it easier than the Semplice team did, Filip and Miloš and the Trios team did, or certainly I did when we began looking at different installers

Here is the How To build and install the Semplice/Trios/VSIDO linstaller for debian:




Build the Installer

First some requirements and dependencies to build

sudo apt-get update && sudo apt-get dist-upgrade

sudo apt-get install python-all screen python-notify2 python-progressbar python-parted imvirt cryptsetup fatresize hfsprogs pastebinit reiser4progs jfsutils debhelper dh-python dpkg-dev devscripts git

Keeptalking build

git clone https://gitlab.com/Cofi/keeptalking.git && cd keeptalking

dpkg-buildpackage -uc -us


Bricks build

git clone https://gitlab.com/Cofi/bricks.git && cd bricks

dpkg-buildpackage -uc -us


Installer build

For a Standard Build:

git clone https://gitlab.com/Cofi/trios-installer.git && cd trios-installer

dpkg-buildpackage -uc -us


For Testing Build: (Can only be Standard or Testing, NOT both)

git clone https://gitlab.com/Cofi/trios-installer.git && cd trios-installer && git checkout testing

dpkg-buildpackage -uc -us





Install the VSIDO Installer

First some requirements or dependencies:


sudo apt-get update && sudo apt-get dist-upgrade

sudo apt-get install python-all screen python-notify2 python-progressbar python-parted imvirt cryptsetup fatresize hfsprogs pastebinit reiserfsprogs jfsutils xfsprogs btrfs-tools


Now install the debs In This Order

cd to each directory these are in (Put them all in one single directory) ... the tab to complete fills out the remaining portion of the file name.. saves time and gets it correct for whatever version is there (version numbers will change)

sudo dpkg -i python-t9n (tab to complete)
sudo dpkg -i python-keeptalking (tab to complete)
sudo dpkg -i python-libbricks (tab to complete)
sudo dpkg -i keeptalking (tab to complete)
sudo dpkg -i bricks (tab to complete)
sudo dpkg -i linstaller-modules-base (tab to complete)
sudo dpkg -i linstaller-modules-bricks (tab to complete)
sudo dpkg -i linstaller-frontend-cli (tab to complete)
sudo dpkg -i linstaller-frontend-glade (tab to complete)
sudo dpkg -i linstaller_ (tab to complete)
sudo dpkg -i linstaller-config-vsido (tab to complete)
sudo dpkg -i linstaller-session (tab to complete)
sudo dpkg -i live-config-linstaller (tab to complete)


After Installation

Now edit these files:

sudo medit /usr/share/linstaller/linstaller/modules/network/inst/_init_.py

and make this

auto lo
iface lo inet loopback
""")


this

auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eth0
iface eth0 inet dhcp
""")



Next create or convert these in /etc/linstaller directory (these must replace the trios files that are in /etc/linstaller directory)

sudo medit /etc/linstaller/vsido

## MAIN LINSTALLER CONFIGURATION FILE FOR VSIDO

[linstaller:extends]
source = vsido-base

# Do not ask for language, use the system's one instead.
#[module:language]
#ask = False

# Do not ask for timezone, use the system's one instead.
#[module:timezone]
#ask = False

# Disable root
#[module:userhost]
#root = None


sudo medit /etc/linstaller/vsido-base

## BASE LINSTALLER CONFIGURATION FILE FOR VSIDO -- END USER WILL NOT (HOPEFULLY) NEED THIS

[linstaller]
distribution = VSIDO
modules = uefidetect.inst welcome.front language.front timezone.front userhost.front partdisks.front bootloader.front summary.front partdisks.inst unsquash.inst virtualpartitions.inst language.inst timezone.inst network.inst supportrepo.inst bootloader.inst userhost.inst debian.inst clean.inst fstab.inst finish.inst end.front
special = partdisks.inst unsquash.inst virtualpartitions.inst

[module:language]
ask = True

[module:timezone]
ask = True
timezone = Etc/GMT

[module:partdisks]
swap_noformat = True

[module:unsquash]
image = /lib/live/mount/medium/live/filesystem.squashfs

[module:debian]
remove = /etc/linstaller/trios.packages-remove
autoremove = True

[module:bootloader]
bootloader = grub
custom_kernel_args = grub_cmdline_linux
should_hide_menu = False

[module:supportrepo]
path = file:///lib/live/mount/medium/support
binarydir = ./



sudo ln -s /etc/linstaller/vsido /etc/linstaller/default

(You may have to delete the original /etc/linstaller/default to complete that command)

Profit!

Thank you to the Semplice team who created the original.  I also am grateful to the Trios development team for the sharing of their wisdom with the best FOSS intentions I have seen in a long time. Specifically I want to thank Filip Danilović and Miloš Pavlović,  people I am honored to consider friends
Title: Re: How To: Building the VSIDO Installer
Post by: Snap on February 04, 2016, 07:14:54 AM
Thanks for this. I don't know if I will ever need to build the installer, but that's precious info and a very good reading.
Title: Re: How To: Building the VSIDO Installer
Post by: misko_2083 on February 04, 2016, 05:27:34 PM
@VastOne I'm glad you are sharing this with everyone. That's what FOSS is all about.
Just noticed dependancies.
Filip has made changes with dependancies. One is a switch from reiser4progs to reiserfsprogs.
Reiser4 and reiserfs are not the same fs. The switch from reiser4 to reiserfs happened because reiser4 kernel module is not mainlined yet, and therefore doesn't exist in the Debian kernel.
One can actually create/format a reiser4 partition, but can't use it without manually compiling the kernel module first.
Filip also added xfsprogs, jfsutils and btrfs-tools. Changes are here
https://gitlab.com/trios-linux/trios-installer/commit/5848e9069a4c90cff5d9ecbc2b201d6904498da0 (https://gitlab.com/trios-linux/trios-installer/commit/5848e9069a4c90cff5d9ecbc2b201d6904498da0)
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on February 04, 2016, 05:35:18 PM
@Misko... correct, this will be an evolving How To as it progresses

Thanks for those updates, I will correct them on the How To though I did already have all the required dependencies since I did build from Testing and had to have them
Title: Re: How To: Building the VSIDO Installer
Post by: filip on February 04, 2016, 10:49:08 PM
@Misko pretty much said it all. :)
Thanks for sharing and for the effort!

Few notes along the way:

@VSUDO config:

To avoid having to change it on every rebuild, you can do it once, build the packages and then tuck away somewhere the "linstaller-config-trios" ( you can change the "trios" to "vsido" in debian/control ).
You can then re-use it, since it doesn't have a versioned dependencies, so should work with any new version of the installer. :)

@UEFI:

You can also delete all those lines, instead of commenting them ( which is a pain whenever there's more than 2-3 of them, for me atleast :D ).

@pkg installation:

A (much) easier way would be to put them all in a script:
echo '#!/bin/bash
sudo dpkg -i python-t9n
sudo dpkg -i python-keeptalking
sudo dpkg -i python-libbricks
sudo dpkg -i keeptalking
sudo dpkg -i bricks
sudo dpkg -i linstaller-modules-base
sudo dpkg -i linstaller-frontend-cli
sudo dpkg -i linstaller-frontend-glade
sudo dpkg -i linstaller
sudo dpkg -i linstaller-config-trios
sudo dpkg -i linstaller-session
sudo dpkg -i live-config-linstaller' >> /tmp/install-installer && chmod +x /tmp/install-installer && /tmp/install-installer

^ copy/paste all at once. :)

Cheers!
Title: Re: How To: Building the VSIDO Installer
Post by: filip on April 08, 2017, 09:38:52 PM
@VastOne:

Updated How-To is spot on, except that the entire "After installation" section is redundant and no longer needed ( since installer v7.0.7~fd2 ).

Also, "bricks" module is not used in VSIDO ( and cannot be used, without modifications ), so you might as well remove it's section to keep things simpler.  :)
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on April 08, 2017, 09:41:33 PM
Yes Filip.. I am testing that right now on the uEFI builds and have already made some changes to the How To

Once I am done with this test I will make the complete changes and build and commit all ISO's for release

Thank you!
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 17, 2018, 04:43:27 PM
The VSIDO Installer is now dead on latest (non released) ISO test builds... It gets to the second screen and stops dead and nothing works to kill it. Some kind of update to SID has killed a dependency for the installer

From terminal the only thing showing is

REBUILDING LOCALE MODEL

Obviously nothing will be released until this is resolved

I'll try to rebuild the installer from a LiveCD session to see if it has been fixed at all from the TRIOS team
Title: Re: How To: Building the VSIDO Installer
Post by: filip on March 19, 2018, 03:24:49 PM
Quote from: VastOne on March 17, 2018, 04:43:27 PM
From terminal the only thing showing is

REBUILDING LOCALE MODEL
Hmm, it's unusual for Python not to output any error(s), so it may actualy be somewhere outside...
But anyhow, it would be most helpful if you could upload the latest build/ISO somewhere for me to download and see what's up!?
Updating the live system is a pain and likely won't produce the same environment as the prebuilt one. :)
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 19, 2018, 03:49:11 PM
Thank you filip..

The latest test build ISO is here (http://www.nixnut.com/vsido/test/)

As you see from the date that is over a week old.. I will also update the build environment to today's level, build a new ISO and test it

That will take me some time, I'll get back to you about it if it changes the situation
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 19, 2018, 08:20:54 PM
I rebuilt today and the same issue is there.. I have uploaded that ISO as well
Title: Re: How To: Building the VSIDO Installer
Post by: filip on March 20, 2018, 03:35:43 PM
"Keeptalking" module is at blame here.
Something is going wrong with locale list building. Probably something changed in Sid/locales making the keeptalking parser read garbage/lock itself in an endless loop.

I'll resume hunting tonight, hopefully having it by the head by the end of the day.  :)
Title: Re: How To: Building the VSIDO Installer
Post by: filip on March 20, 2018, 04:12:01 PM
Meh, got it.  8) 
"/usr/share/i18n/cns11643_stroke" file appeared probably recently in Sid, causing "keeptalking" to stumble over it. It is now properly ignored (https://gitlab.com/Cofi/keeptalking/commit/5cb55d5aaf6b0864f8a7336bb7dccc322ab129bb), so the issue should be fixed.
All you need to do is build the updated "keeptalking" package(s), version 6.0.3~trios1, using the usual mantra, git clone --> dpkg-buildpackage...
https://gitlab.com/Cofi/keeptalking
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 20, 2018, 10:41:11 PM
Hi Filip..  There seems to be issues with the 6.0.3 version of the keeptalking.. I built it and installed it with no issues but had the same results on the first build of the ISO.  I confirmed from that livcd session that the keeptalking/Locale/__init__.py had the original and did not have the fix:

if "translit" in _file or "iso14651_t1" in _file or "POSIX" in _file or "cns11643_stroke" in _file: continue

I then checked it also in the build environment and confirmed that both 6.0.3 was installed and the fix was not there either

I manually edited the __init__.py file and added the changes, rebuilt the ISO and it the install worked flawlessly

Need anything else, please let me know
Title: Re: How To: Building the VSIDO Installer
Post by: filip on March 22, 2018, 07:13:08 PM
Quote from: VastOne on March 20, 2018, 10:41:11 PM
Hi Filip..  There seems to be issues with the 6.0.3 version of the keeptalking.. I built it and installed it with no issues but had the same results on the first build of the ISO.  I confirmed from that livcd session that the keeptalking/Locale/__init__.py had the original and did not have the fix:

if "translit" in _file or "iso14651_t1" in _file or "POSIX" in _file or "cns11643_stroke" in _file: continue

I then checked it also in the build environment and confirmed that both 6.0.3 was installed and the fix was not there either

I manually edited the __init__.py file and added the changes, rebuilt the ISO and it the install worked flawlessly

Need anything else, please let me know
It's most likely has something to do with the way you cloned git/built the package. I've just tested with the fresh clone, and the package (python-keeptalking) has the latest __init.py__ in question.

Steps I've followed ( on Stretch though, the issue you're having might also be Sid related ):git clone https://gitlab.com/Cofi/keeptalking.git && cd keeptalking

Verify that we're on "trios-master" branch:git branch  #should output: "* trios-master"

Double check the fix:grep cns11643 ./keeptalking/Locale/__init__.py
#output:                  if "translit" in _file or "iso14651_t1" in _file or "POSIX" in _file or "cns11643_stroke" in _file: continue


Build:dpkg-buildpackage -uc -us

Result (gdebi):
  (http://imgur.com/5XafyH4l.png)
(http://i.imgur.com/5XafyH4.png)
:)

Also, do make sure that you actually included the 6.0.3 version of the "python-keeptalking" package in the ISO.
The reason I'm even mentioning this is that "keeptalking" package doesn't have versioned dependency on it, so you could install new "keeptalking" and keep the old "python-keeptalking" without apt complaining ( I should really fix this  :D )...
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 22, 2018, 10:21:55 PM
I did everything exactly as you did in git, the build and made sure it was 6.0.3 in the build..

Running apt-cache policy keeptalking

clearly shows 6.0.3~trios1 is installed in the build and in the installs

git branch = trios-master and the grep output is exactly the same:

grep cns11643 ./keeptalking/Locale/__init__.py
if "translit" in _file or "iso14651_t1" in _file or "POSIX" in _file or "cns11643_stroke" in _file: continue


I have attached the built .deb file created and what I used ... in looking at the included file keeptalking_6.0.3~trios1_all.deb (in gdebi and xarchiver) I do NOT see a Locale directory or the __init__.py file

I have also attached the keeptalking_6.0.3~trios1.tar.xz file and this DOES have the Locale directory and the __init__.py file

This is a strange one.. I'm not sure what has happened but it has...  ::)
Title: Re: How To: Building the VSIDO Installer
Post by: filip on March 23, 2018, 01:45:14 PM
Quote from: VastOne on March 22, 2018, 10:21:55 PM
I have attached the built .deb file created and what I used ... in looking at the included file keeptalking_6.0.3~trios1_all.deb (in gdebi and xarchiver) I do NOT see a Locale directory or the __init__.py file

I have also attached the keeptalking_6.0.3~trios1.tar.xz file and this DOES have the Locale directory and the __init__.py file

This is a strange one.. I'm not sure what has happened but it has...  ::)
Keeptalking source produces three packages, "keeptalking", "python-keeptalking" and "python-t9n":
[filip@stretch-K10][/tmp/keeptalking]$ ls ../ | grep .deb
keeptalking_6.0.3~trios1_all.deb
python-keeptalking_6.0.3~trios1_all.deb
python-t9n_6.0.3~trios1_all.deb
And "/Locale/__init__.py" belongs in the "python-keeptalking".  ;)
Thus, you need all three installed and at the same version. I'd bet that "apt-cache policy python-keeptalking" shows it's an older version compared to "keeptalking"!?

Like I've said previously, "keeptalking" should really have versioned depends on the other two, so apt would complain if there's a version mismatch between any of them.
In other words, if you have "python-keeptalking" v=6.0.0 installed, and try to update only "keeptalking" to v=6.0.3 apt would complain and requre you to update "python-keeptalking" and "python-t9n" to v=6.0.3 as well.

Anyhow that's fixed (https://gitlab.com/Cofi/keeptalking/commit/ef0433f4fe4e91861ea3c4c1cf76065b2b1fd335) in latest git (v=6.0.3~trios2).  :)

So, clone --> dpkg-buildpackage and then:cd ../ && sudo apt install ./keeptalking_6.0.3~trios2_all.deb ./python-keeptalking_6.0.3~trios2_all.deb ./python-t9n_6.0.3~trios2_all.deb
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 23, 2018, 10:46:34 PM
^ Thanks man.. sorry I missed that on the earlier post... and all those deb files are there I just saw the first keeptalking deb and installed that and never looked past my mistake

Sorry that I was so obtuse Filip.. I appreciate your help on this

It's funny how most issues are staring you right in the face and you don't see them
Title: Re: How To: Building the VSIDO Installer
Post by: filip on March 24, 2018, 06:51:19 PM
Quote from: VastOne on March 23, 2018, 10:46:34 PM
^ Thanks man.. sorry I missed that on the earlier post... and all those deb files are there I just saw the first keeptalking deb and installed that and never looked past my mistake

Sorry that I was so obtuse Filip.. I appreciate your help on this
No problem at all!  :)

Quote from: VastOne on March 23, 2018, 10:46:34 PM
It's funny how most issues are staring you right in the face and you don't see them

I´d say it´s not much of a problem per se, however the frequency at which it occurs.....  :D ;D
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 16, 2019, 12:10:49 PM
Hey Filip I hope all is well with you

I am getting the following error trying to build the latest

dpkg-buildpackage -uc -us
dpkg-buildpackage: error: cannot open file debian/changelog: No such file or directory


It looks like it may be due to Debian changing it's folder names from debian to debianpkg?

Just spitballing, I am not sure and Google isn't much help

Thanks
Title: Re: How To: Building the VSIDO Installer
Post by: filip on March 17, 2019, 01:44:01 PM
Hey,

Oh yeah, still kicking   8) ;D Albeit very little in the foss world :(

I'll try it out in a VM tonight/tomorrow, but before that, are you sure that you're in the right directory? That error usually comes out if your current dir is not one above "debian".
If the dir is correct, please post the "ls -la" output from there.


Didn't hear anything about "debian --> debianpkg", Google finds nothing, but anyway I doubt they'd make such a profound change now that the Buster freeze is ongoing.Which btw, has the coolest name so far  ;D


Cheers

Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 17, 2019, 01:59:24 PM
EDITED

It doesn't matter at this point anyway filip.. nothing has changed in the installer that would address the issue we are having covered here (https://vsido.org/index.php?topic=1395.msg15957#msg15957) where no drives are showing up to be installed to in the gparted section unless you completely wipe everything before installing
Title: Re: How To: Building the VSIDO Installer
Post by: filip on March 17, 2019, 02:15:42 PM
No "debian"  dir there. Maybe something went sideways with the "git clone" or you forgot to switch to "trios-master" branch?? :)
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 17, 2019, 02:16:59 PM
See my edit please.. 
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 17, 2019, 02:17:47 PM
I ended up  having the files after all and they have not changed
Title: Re: How To: Building the VSIDO Installer
Post by: filip on March 17, 2019, 02:21:03 PM
Quote from: VastOne on March 17, 2019, 02:17:47 PM
I ended up  having the files after all and they have not changed
I'm not sure what are you saying? It builds now? :)
@EDIT:
I've looked at the posts about april ISO... Anyway, can't tell you much until I take a look in the VM. Will get back when I do :)
Title: Re: How To: Building the VSIDO Installer
Post by: VastOne on March 17, 2019, 02:23:03 PM
I am saying I have what I need to check on build levels and nothing new has happened in development..

Sorry for the mass confusion

In the end I think this is a kernel issue but I cannot confirm it yet