TrochejEN

illumian - debian-like distribution of illumos

2012-02-13T05:57:00.000-08:00

Recently my attention got caught by illumian, a Debian-like distribution of illumos. Since I come from Debian world, having administered Debian based servers for over ten years now, this distribution that uses .deb package format caught my eye. While I slowly get used to IPS, I find it far less friendly than apt- and friends.
What is this illumian distribution? It's basically OpenIndiana with IPS exchanged in favor of .deb packages. ISO file that you can grab from there contains installation media for server solution, which means that if you want to get graphical environment working, you have to install X and GNOME packages yourself. In short it comes down to installing x11 and gnome packages and enabling gdm service via svcadm.
Have a nice testing. :)

2012-02-07T02:25:00.000-08:00

Experiences Starting an OpenSource Operating System. Garrett D'Amore speaks at SCALE10x about the genesis of illumos. Recorded by Deirdré Straughan on her Youtube channel.

OpenIndiana and ntfs-3g

2011-11-02T14:07:00.000-07:00

Update:
I have patch ready to change unhelpful on OpenIndiana, SmartOS and other illumos distributions mention of modprobe to suggest:
/usr/sbin/add_drv -m 'fuse 0666 root sys' fuse
Still have to find out, why the heck link was not created in the first place. :/

----------------------------------
I recently had to use ntfs formatted USB drive under OpenIndiana and run into a small problem.
I shamelessly used gparted to find which device maps to my usb disk. Then had following session with ntfs-3g binary:

ntfs-3g /dev/dsk/c2t0d0p1 /mnt/
The disk contains an unclean file system (0, 0).
The file system wasn't safely closed on Windows. Fixing.
fuse: device not found, try 'modprobe fuse' first

Both fuse and ntfs-3g are installed from repositories:
sfe                                   origin   online   http://pkg.openindiana.org/sfe/
sfe-encumbered                        origin   online   http://pkg.openindiana.org/sfe-encumbered/

After some digging around and asking on IRC I decided that either no one had this problem or no one bothered to post solution. modinfo showed me that fuse module is loaded, after all:

258 fffffffff7edd000   e680 290   1 fuse (fuse driver)
258 fffffffff7edd000   e680 28   1 fuse (filesystem for fuse)

However ntfs-3g itself didn't say anything useful. It also suggested to use modproble, which is just ridiculous on OpenIndiana. I decided to truss it.

open64("/dev/fuse", O_RDWR)       Err#2 ENOENT

One clue closed I downloaded spec file for fuse from http://pkgbuild.sourceforge.net/spec-files-extra/ and checked for %post step:

%post
( retval=0;
/usr/sbin/add_drv -m 'fuse 0666 root sys' fuse || retval=1;
[ "$retval" = 0 ] && ln -s /devices/pseudo/fuse@0:fuse /dev/fuse || retval=1;
exit $retval
)

I did check if /devices/pseudo/fuse exists (it did) and then created /dev/fuse link to it. Then ntfs-3g worked like a charm.

Cheers!

ZFS is ten

2011-11-02T05:42:00.000-07:00

ZFS turned ten years old. There is short but nice blog post about it. One thing worth noting is, how long it takes a feature from initial implementation to first introduction to operating system. Four years before inclusion in OpenSolaris project and then in Solaris 10 in 2006 IIRC.

http://blog.delphix.com/matt/2011/11/01/zfs-10-year-anniversary/

OpenIndiana 151a release

2011-09-13T23:27:00.000-07:00

OpenIndiana project has just released stable 151a release.
Follow download link for the goodness. There are also instructions for upgrading from previous OpenIndiana releases and from OpenSolaris.

illumos based OpenIndiana

2011-02-23T03:54:00.001-08:00

Update 26.02.2011.
There is also a USB image. You can use it, quite conveniently, with dd tool on Linux or with OpenSolaris USB Live creator on Windows written some time ago by Hiroshi Chonan.
To write it to USB stick with dd, you have to prepend a special header, which you can download here. Just do:

wget http://dlc.openindiana.org/isos/2G.header
wget http://dlc.openindiana.org/isos/148/oi-dev-148a-x86-20110223-1.usb
cat 2G.header oi-dev-148a-x86-20110223-1.usb >> oi-dev-148a-x86-20110223-1-header-prepended.usb
dd if=./oi-dev-148a-x86-20110223-1-header-prepended.usb of=/path/to/your/usb/device (ie. /dev/sdb)

Of course, with OpenSolaris based distributions, you can use /usr/bin/usbcopy from pkg:/install/distribution-constructor

Cheers!
-------------------
And so it happened. OpenIndiana has now published illumos based authority and isos.
Authority: http://pkg.openindiana.org/dev-il/ - based on illumos changeset fe7962c08d1d; the latest changeset in illumos repository.
Isos: http://dlc.openindiana.org/isos/148/oi-dev-148a-x86-20110223-1.iso

Illumost testing build

2011-01-18T02:49:00.001-08:00

UPDATE: Installation instructions corrected. Thanx aszeszo for that. :)
Last week I've built packages from illumos sources, changeset cac385f011a3. Project OpenIndiana was kind to set up a repository with packages that you can use to upgrade your OpenIndiana system:

http://pkg.openindiana.org/illumos-experimental/
General instructions follow:

pkg set-publisher --non-sticky openindiana.org
pkg set-publisher -P -O http://pkg.openindiana.org/illumos-experimental/ on-nightly
pkg uninstall entire
pkg image-update -v --be-name illumos-1

Enjoy.

find and symlinks

2010-10-20T04:25:00.000-07:00

A pearl of wisdom from today problems: find does not automatically descend into symlinks to directories. You have to ask it to, by using -follow. It turns out that trivia is not widely known at this site.
This has today surfaced with filesystem hierarchy resembling below:

drwxr-xr-x /usr/local/bin/Testing
lrwxrwxrwx /opt/Testing -> /usr/local/bin/Testing

It translates into: /usr/locla/bin/Testing is a directory and /opt/Testing is a symbolic link to this directory. Within this directory lies a file test.t.
A developer was bewildered why, when he cds into /opt/Testing, following find the file:

find . -name test.t

and following does not:

find /opt/ -name test.t

Reason is simple. find, by default, will not follow symlinks to directories. There is an option for that: -follow.

Closed crypto removed from illumos - FLAG DAY

2010-09-13T00:24:00.001-07:00

Hi

I paste e-mail from Garrett, which constitutes a flag day for everyone building illumos nightly. Please, read it carefully, as it explains itself nicely:

With my integration of "6 Need open kcfd", there is actually no more
kcfd or signed crypto tarballs. (The necessary code for thread
management is in the kernel kcf. You'll notice new kernel LWPs called
"kcfpoold" in ps -eLf output.) Also, all vestiges of FIPS-140 support
are removed.

This represents a flag day for everyone building illumos.

You must do a full nightly across thsi flag day -- incrementals *will*
break.

You must not have a crypto tarball. (One less thing to download from
Oracle.. yay!)

You *MUST* use the version of nightly in the latest source. It is a
syntax error to try to use the old nightly -- it won't work unless you
have the old crypto tarball, and if you do, the result will not be
functional!

To recap:

* remove crypto tarball if present.
* use nightly from source tree
* do a full clobber build.

Thank you.

- Garrett

How to build illumos guide has been updated to follow this change. Enjoy!

Broken splashimage with illumos boot menu

2010-09-01T04:33:00.000-07:00

Due to some misunderstanding, a solaris.xpm have been removed too early from illumos source tree. It results in problems while trying to boot your new illumos boot environment, if you have updated with onu.sh script.
If you happen to have this problem, what you need is modify your grub menu. Mine looks like this.

title os-devel_145
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/os-devel_145
splashimage /boot/solaris.xpm
foreground d25f00
background 115d93
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS,console=graphics
module$ /platform/i86pc/$ISADIR/boot_archive

What you need to do, is edit it, so it looks like below:

title os-devel_145
findroot (pool_rpool,0,a)
bootfs rpool/ROOT/os-devel_145
kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS
module$ /platform/i86pc/$ISADIR/boot_archive

On somewhat related note, booting to single user mode is done with adding -s at the end of the kernel$ line:

kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS -s

The problem with missing splashimage is known and a bug has been filed.

Illumos building instructions

2010-08-29T22:57:00.000-07:00

As far as we know, Oracle has stopped developing OpenSolaris in the open. For those of us, who would rather have Opensolaris become a real openly developed operating system, ptoject illumos came to life. Thanks to Garrett d'Amore et consortes. On the illumos wiki page there is a nice instruction on compiling your own post build 134 illumos source.
Already much work has been put in the source tree to liberate it from dependency on /extra repository and new putbacks arrive all the time.
If you'd like to check it for yourself, here is the link to How to build illumos guide.

Zones delegated administration

2010-06-15T00:13:00.001-07:00

To counter my previous entry, this is something that touches Zones also, but is pretty nice feature:
http://static.opensolaris.org/on/flagdays/pages/20100607142839.html - delegated zones administration.
Basically a user or role in global zone can be designated in a zone configuration as a admin resource, thus giving the user/role additional abilities to control zones.
A good example taken from the link above:

           example# zonecfg -z myzone
          zonecfg:myzone> add admin
          zonecfg:myzone:admin> set user=zadmin
          zonecfg:myzone:admin> set auths=login,manage,clonefrom
          zonecfg:myzone:admin> end
          zonecfg:myzone> commit


No, user zadmin can use profiled shell to administer zones.

Linux Brandz removed from ON

2010-06-15T00:00:00.000-07:00

ONNV build 143 is a important for every one of us that uses lx Branded Zones - the Linux ones. There have been a stall in development since some time and forums are full of people looking for ways to install anything newer than CentOS images provided by the project. But since the build 143 Linux Brandz will be removed. If you you is too, like me, you have to plan for some migration.
And so Linux Brandz go away: http://static.opensolaris.org/on/flagdays/pages/20100612010403.html

SFE bootstrap script

2010-06-07T23:19:00.000-07:00

Thanks to Tommw, who maintains sfe bootstrap script, you can set up a complete SFE compilation ready environment in few simple moves.
The script can be downloaded from here: http://tinyurl.com/bootstrap-sfe-latest-os20nn. Just use wget:

wget -N "http://tinyurl.com/bootstrap-sfe-latest-os20nn"

chmod a+rx bootstrap-sfe-latest-os20nn

./bootstrap-sfe-latest-os20nn username

Username is a login of a user that you'll use for compilation. The script does all the magic, just answer two or three simple questions - user's password.
Before you begin, you may want to:

Create a user that will be responsible for compilation. User should have a "Software Installation" profile assigned. I use my main user, who has both Software Installation and Primary Administrator profiles.

pfexec pkg install wget

Enable local repository, as pkgbuild will want to push packges to your IPS repository. This is a link to repository setup instructions. Go on, it's friggin easy.

Remember to send Thanks e-mail to address printed after the bootstrap script finishes.

PSARC 2009/377; 2009/378; 2010/072

2010-04-28T03:43:00.000-07:00

PSARC 2009/377

In-kernel pfexec implementation.

PSARC 2009/378

Basic File Privileges

PSARC 2010/072

RBAC update: user attrs from profiles,

4912090 pfzsh(1) should exist,

4912093 pfbash(1) should exist,

4912096 pftcsh(1) should exist,

6440298 Expand the basic privilege set in order to restrict file access,
6859862 Move pfexec into the kernel, [that one is interesting - trochej]

6919171 cred_t sidesteps kmem_debug; we need to be able to detect bad hold/

pfexec is a kind of sudo, right?

2009-08-14T00:39:00.000-07:00

(Open)Solaris can cause some emotional turmoil in new users coming from Linux background. Its ifconfig works bit differently, load average is calculated differently, and so on, et caetera...
One of things that leave people rather puzzled is authorisation mechanism based on system roles: Role Based Access Control (RBAC). As always, you're encouraged to read fantastic docyumentation: System Administration Guide: Security Services, chapterł Roles, Rights Profiles, and Privileges. However, to make a very short introduction, RBAC defines roles and profiles that have assigned privileges (authorisations) regarding use of commands. Something like sudo, at the first sight. Those roles and profiles can be assigned to concrete users. Something like sudo. The first difference is simply cosmetic. Instead of invoking 'sudo' in front of privileged command, one invokes 'pfexec'. The second difference is strictly work based. RBAC (pfexec) comes with multitude of profiles and roles ready to be used. For last few years I used OpenSolaris I prepared new profile just once.
Third difference lays in profiled shells, ie. pfsh. If a user is assigned a profile and given a profiled shell, they will be able to invoke only commands which are assigned to said profile, but without using the pfexec keyword.
Fourth difference are roles. Roles are basically user accounts that cannot be logged into directly. One has to use su command to log into the roles. Also, user can only logged into the role which has been assigned to them. This is the default use of root account in OpenSolaris distribution.
Fifth difference is big. RBAC reaches deeply in structure of (Open)Solaris, down to special kernel structures. This, along with rbac programming libraries, allows to implement very fine grained security into programs written. So, a fragment of code can check, if invoking user has, say, authorisation to write to log files, and then do some action.

Anatomy.

authorisations - users rights to use privileged functions,

execution profiles - a way of combining authorisations and command with attributes, ie. UID of command,

roles - accounts explained before

Authorisations are defined in a system file /etc/security/auth_attr.
Users get profiles, roles and authorisations assigned in a file /etc/user_attr. Some fun fact. Some time ago blogs and howtos instructed that to assign a role, profile or authorisation to user, one had to edit said file. However, it is officially discouraged way. Better way is to use commands. Commands are guaranteed not to break the file. It is important to understand, that broken profile in /etc/user_attr may render user unable to user any privileged commands, login to root account and so on.
Authorisation definitions are written in a file /etc/security/prof_attr.
And the most important, I think - definitions of privileged operations assigned to profiles are written in a file /etc/security/exec_attr.
I won't indulge in details, as those are well explained in documentation (man man, people!). I'll only mention, that commands assigned to security profile are listed in /etc/security/exec_attr. A bit mysterious is file /etc/security/auth_attr. When I studied RBAC I was puzzled. All other files are intertwined together and it is rather easy to trace how they work together, but I couldn't figure out this one, as I couldn't find any sign of listed there attributes any other place. It turns out, that this is the mechanism for implementing rbac in applications. A code can do a check against any authorisation listed in the file and if a user is assigned one, application may take certain actions.

Commands
Managing RBAC is nased on few commands.

roles - prints list of roles assigned to a user,

profiles - prints a list of users profiles. profiles -l prints authorisations and privileged commands of users every profile,

auths - prints list of users authorisations,

roleadd - creates a role, syntax similar to useradd,

rolemod - modifies a role,

useradd: option -P accepts list of profiles as its argument, option -R accepts list of roles, option -A accepts list of athorisations,

usermod: option -P accepts list of profiles as its argument, option -R accepts list of roles, option -A accepts list of athorisations.

WARNING!
rolemod and usermod overwrite assigned privileges. If one wants to assign a profile Software Installation to user that already has Primary Administrator, usermod line looks like that:

pfexec usermod -P "Primary Administrator,Software Installation" username

More information in manpages and : System Administration Guide: Security Services, chapter Roles, Rights Profiles, and Privileges.

beadm

2008-05-15T13:33:00.000-07:00

In my last post I wrote a short howto to handmade zfs rpool cloning. It takes only a few steps, but forgetting one of them may provide wrong results. But OpenSolaris has one nice tool to do it all for you:


beadm create opensolaris-2

Simply do it, and your system will be cloned, your grub will have a new entry and the new filesystem's /etc/vfstab will have it's '/' entry corrected.
Now, why not do a clone every few days, just in case you ever remove your '/etc'?

Playing with fire

2008-05-13T21:52:00.000-07:00

Following losely Indiana update guidelines found here I have done a clone of my root filesystem. Then I have fooled around with it a little bit.


zfs snapshot rpool/ROOT/opensolairs@for-cloning
zfs clone rpool/ROOT/opensolaris@for-cloning rpool/ROOT/opensolaris-1
zfs set mountpoint=legacy rpool/ROOT/opensolaris-1
mount -F zfs rpool/ROOT/opensolaris-1 /mnt
vim /mnt/etc/vfstab

Here I have replaced rpool/ROOT/opensolaris with rpool/ROOT/opensolaris-1. I yet edited /rpool/boot/grub/menu.lst by copying an OpenSolaris entry and changing the bootfs rpool/ROOT/opensolaris line in the copy to bootfs rpool/ROOT/opensolaris-1. In the end I had two different entries in grub boot menu.
I then did:


rm -rf /platform/

and rebooted. In the above one line I have effectively wiped out kernel and boot_archive. It came as no surprise, that grub complained that it couldn't boot my old system. Oh well, I choose the second entry, booted from the clone, promoted it with zfs promote, destroyed the old system, which became the clone now, and type to you quite happily from working OpenSolaris 2008.05.
Have a Nice day.

ZFS snapshots

2008-05-09T13:36:00.001-07:00

I started to like them. I became accustomed to create a snapshot of my filesystems periodically, just as I click Save button in Editors. Really nice feature, which makes my life easier. Undelete is almost obsolete. :)
For interested:


-bash-3.2# cat /usr/bin/zfs_snapshot.sh
#!/bin/sh

zfs snapshot -r rpool@`date +%Y-%m-%d:-:%H:%M`


-bash-3.2# time zfs_snapshot.sh

real 0m1.257s
user 0m0.014s
sys 0m0.023s

-bash-3.2# zfs list
NAME                                              USED  AVAIL  REFER  MOUNTPOINT
rpool                                            21,2G  67,9G  56,5K  /rpool
rpool/ROOT                                       5,94G  67,9G    18K  /rpool/ROOT
rpool/ROOT/opensolaris-1                         5,94G  67,9G  5,28G  legacy
rpool/ROOT/opensolaris-1@2008-05-09:-:22:59          0      -  5,28G  -
rpool/ROOT/opensolaris-1/opt                      673M  67,9G   673M  /opt
rpool/ROOT/opensolaris-1/opt@2008-05-09:-:22:59      0      -   673M  -
rpool/export                                     15,2G  67,9G    19K  /export
rpool/export/home                                15,2G  67,9G  15,2G  /export/home
rpool/export/home@2008-05-09:-:22:59             70,5K      -  15,2G  -

prstat

2008-03-19T13:24:00.000-07:00

I wrote about it in my Polish blog, but have seen it a few times since then on English channels that I decided to post it here also.
A person at #opensolaris channel had a problem, that


prstat -m | nawk '{print $10}' | nawk '{sum+=$1;} END {print sum;}'

just happily sat there printing nothing. It helps to know, that prstat, as the last arguments, takes interval between measurements and a count of measurements. If the count is omitted, prstat will take measurements waiting for ^C, thus the END will never be satisfied.
The version below works, taking one measurement and then printing the output.

prstat -m 1 1 | nawk '{print $10}' | nawk '{sum+=$1;} END {print sum;}'

One more thing worth knowing is that prstat, similarly to other tools, as a first measurement prints averages since system boot. Next measurements are closer to the actual state of system.