piątek, 7 grudnia 2012

Garrett D'Amore: [developer] webrev: pfile postmortem support

UPDATE:
Just noticed in repository that this work has been integrated:

changeset:   13875:f128a109e6d2
user:        Garrett D'Amore
date:        Fri Nov 02 09:48:42 2012 -0700
summary:     3294 pfiles postmortem support

Sorry that it goes through my blog, but I can't work through listbox weird URLs.
This is bit dated, as the thread started back in October. There is an interesting thread going just after this mail with reviews, but if you want to read it, go to listbox archive and search for "webrev: pfile postmortem support".
Below a quote from Garrett's mail that explains everything cleanly:

From: Garrett D'Amore 
Subject: [developer] webrev: pfile postmortem support
To: "developer@lists.illumos.org" 

http://cr.illumos.org/~gdamore/pfiles/

This adds the ability to use pfiles to get file information on a core dump.=
  To support this, I've had to create a new elf note, NT_FDINFO, and a stru=
cture for it (prfdinfo_t), which contains a bunch of information about open=
 file descriptors.  I've also added support for dumping this note into elfd=
ump.

This is the result of work I began at the illumos hackathon earlier in the =
month, and I've finally had time to get it to the point where I think its r=
eady for people to try, and also review it.  I think its fairly fully baked=
.  It works with both core files generate via gcore, and kernel generated c=
ore files.  (The two core dump generators are totally different code, addin=
g substantially to the effort.  Sadly I've had to teach elfexec far more ab=
out the kernel's user and file structures to support this than I'd like =85=
 the lack of good interfaces to those structures is an area of weakness in =
our current kernel subsystems.)

One thing that this doesn't do is go into more detail about sockets, doors,=
 etc.  I figure that would be a future RFE to add yet additional note secti=
ons (or possibly extending the NT_FDINFO with more information).  I think e=
ven without that information, the current data (which contains details abou=
t path, mode bits, open flags, etc.) is still very helpful for folks doing =
post-mortem debugging.  In particular, this is kernel state that is typical=
ly not available to folks debugging with traditional tools like gdb, dbx, o=
r even mdb.  (A cool follow on project might be to give those debuggers the=
 ability to parse this new note as well.  I'll leave it for someone more fa=
miliar with those tools.)

I've tested this pretty well on x86 systems, but I've not really tested thi=
s on SPARC.  Full testing should involve applying the patch, rebooting, and=
 then generating and testing cores  as well as live procfs attach based tes=
ts for pfiles.  Also, the new elfdump utility should be tested on sparc as =
well.  I'd be happy to hear about any one who wants to take the patch file =
above, test it out on sparc, and let me know.

The review is kind of juicy -- about 900 lines or so.  I'd really like to g=
et this integrated before *too* long though, so I'd appreciate meaningful a=
nd timely review feedback.  If you are going to plan on reviewing it, pleas=
e let me know, and give me an estimate of how long you'd like to review it.=
  Thanks!

Also, any testing (esp. sparc, but not *just* sparc) would be gratefully re=
ceived.

 - Garrett



-------------------------------------------
illumos-developer
Archives: https://www.listbox.com/member/archive/182179/=3Dnow
RSS Feed: https://www.listbox.com/member/archive/rss/182179/22406080-a3f15b=
b4
Modify Your Subscription: https://www.listbox.com/member/?member_id=3D22406=
080&id_secret=3D22406080-849724b2
Powered by Listbox: http://www.listbox.com

czwartek, 22 marca 2012

joyent creates new filesystem for illumos

Yesterday estibi posted a link on irc:
https://github.com/joyent/illumos-joyent/blob/master/usr/src/uts/common/fs/hyprlofs/hyprlofs_vfsops.c#L25
It points to a new filesystem for illumos, a hybrid between lofs (for mounting iso images and other virtual media  filesystem that allows you to make a subtree of one filesystem appear in another -- e.g. in the case of readonly mounts of /usr and /lib into zones - thnx Joshua for correcting me) and tmpfs (in RAM filesystem). Source code explains:
The purpose is to create a high performance name space for files on which
 applications will compute.
 I'm waiting eagerly for joyent to further explain and post example use. You rock, guys!

wtorek, 28 lutego 2012

illumos NTFS-3g kernel module again

Today I saw an e-mail from Jean-Pierre André. He is very active around illumos NTFS-3g kernel module. Please use and test.
Link to Jean-Pierre André's e-mail.
Content pasted below for convenient read.
If you'd like OpenIndiana to support NTFS out of the box, please comment on https://www.illumos.org/issues/1721

Hi,

I am now releasing the fuse kernel module for OpenIndiana. Ntfs-3g
now fully passes the standard tests with this kernel module and without
need of the workarounds I had to insert earlier to cope with the bad
behavior of the fuse kernel module originated from OpenSolaris.

A few optimizations would still be useful. I might have a look at them
if there is enough demand.

Available on http://b.andre.pagesperso-orange.fr/openindiana-ntfs-3g.html
are three packages ready for use :

- a full ntfs-3g package in 32-bit mode with the fuse-lite library
  and ntfsprogs
- a full ntfs-3g package in 64-bit mode with the fuse-lite library
  and ntfsprogs
- a fuse kernel module package for both the 32-bit and 64-bit modes

The raw source files (not packaged according to OpenIndiana
standards) are also available there.

I am keeping these files available on line for some time. I also keep
the change sets available to whoever enters the source code of the
fuse kernel module into a public source code management
repository.

Enjoy,

Jean-Pierre

poniedziałek, 13 lutego 2012

illumian - debian-like distribution of illumos

Recently my attention got caught by illumian, a Debian-like distribution of illumos. Since I come from Debian world, having administered Debian based servers for over ten years now, this distribution that uses .deb package format caught my eye. While I slowly get used to IPS, I find it far less friendly than apt- and friends.
What is this illumian distribution? It's basically OpenIndiana with IPS exchanged in favor of .deb packages. ISO file that you can grab from there contains installation media for server solution, which means that if you want to get graphical environment working, you have to install X and GNOME packages yourself. In short it comes down to installing x11 and gnome packages and enabling gdm service via svcadm.
Have a nice testing. :)

wtorek, 7 lutego 2012

Experiences Starting an OpenSource Operating System. Garrett D'Amore speaks at SCALE10x about the genesis of illumos. Recorded by Deirdré Straughan on her Youtube channel.

środa, 2 listopada 2011

OpenIndiana and ntfs-3g

Update:
I have patch ready to change unhelpful on OpenIndiana, SmartOS and other illumos distributions mention of modprobe to suggest:
/usr/sbin/add_drv -m 'fuse 0666 root sys' fuse
Still have to find out, why the heck link was not created in the first place. :/


----------------------------------
I recently had to use ntfs formatted USB drive under OpenIndiana and run into a small problem.
I shamelessly used gparted to find which device maps to my usb disk. Then had following session with ntfs-3g binary:

ntfs-3g /dev/dsk/c2t0d0p1 /mnt/
The disk contains an unclean file system (0, 0).
The file system wasn't safely closed on Windows. Fixing.
fuse: device not found, try 'modprobe fuse' first

Both fuse and ntfs-3g are installed from repositories:
sfe                                   origin   online   http://pkg.openindiana.org/sfe/
sfe-encumbered                        origin   online   http://pkg.openindiana.org/sfe-encumbered/


After some digging around and asking on IRC I decided that either no one had this problem or no one bothered to post solution. modinfo showed me that fuse module is loaded, after all:

258 fffffffff7edd000   e680 290   1  fuse (fuse driver)
258 fffffffff7edd000   e680  28   1  fuse (filesystem for fuse)


However ntfs-3g itself didn't say anything useful. It also suggested to use modproble, which is just ridiculous on OpenIndiana. I decided to truss it.


open64("/dev/fuse", O_RDWR)       Err#2 ENOENT


One clue closed I downloaded spec file for fuse from http://pkgbuild.sourceforge.net/spec-files-extra/ and checked for %post step:

%post
( retval=0;
  /usr/sbin/add_drv -m 'fuse 0666 root sys' fuse || retval=1;
  [ "$retval" = 0 ] && ln -s /devices/pseudo/fuse@0:fuse /dev/fuse || retval=1;
  exit $retval
)


I did check if /devices/pseudo/fuse exists (it did) and then created /dev/fuse link to it. Then ntfs-3g worked like a charm.

Cheers!

ZFS is ten

ZFS turned ten years old. There is short but nice blog post about it. One thing worth noting is, how long it takes  a feature from initial implementation to first introduction to operating system. Four years before inclusion in OpenSolaris project and then in Solaris 10 in 2006 IIRC.

http://blog.delphix.com/matt/2011/11/01/zfs-10-year-anniversary/