To timidly go where many have gone before

A couple of times in the past, I've spent some time tracking down exactly what scripts/programs are called during Jumpstart and where certain bits of information are determined, but that information is scattered around on bits of paper, in email, and in files that I've lost track of.

Right now, what I'm looking for is how Jumpstart determines what disk to use when you specify "rootdisk" in a profile. For example if you have the following:

partitioning    explicit
filesys         rootdisk.s0      8192   /               logging
filesys         rootdisk.s1      8192   /var            logging
[ ... ]

Jumpstart will pick a disk to use as the root disk and then use that choice consistently through the rest of the process (via $SI_ROOTDISK.) (Or at least it's supposed to do so consistently. I just ran across this. I've never seen the problem, but apparently someone else has.)

The code for determining this is in the chkprobe script (e.g., /export/jumpstart/5.10-sparc-6_06/Solaris_10/Tools/Boot/usr/sbin/install.d/chkprobe.) This is what should happen in most cases:

        if [ -z "${SI_ROOTDISK}" ] ; then
                for i in /dev/dsk/*s0 ; do
                    SI_CDDEVICE=`basename ${i}`
                    findcd -t ${SI_CDDEVICE} > /dev/null 2>&1
                    if [ $? != 0 ] ; then
                        SI_ROOTDISK=${SI_CDDEVICE}
                        break;
                    fi
                done
        fi

Given the ordering presented by the glob, this essentially means that the lowest-numbered disk on the lowest-numbered target on the lowest-numbered controller will be used as the root disk. What's interesting is this bit of code before the above section:

        # see if there is a device c0t3d0s0
        if [ -z "${SI_ROOTDISK}" -a -b /dev/dsk/c0t3d0s0 ] ; then
            findcd -t c0t3d0s0 > /dev/null 2>&1
            if [ $? != 0 ] ; then
                SI_ROOTDISK=c0t3d0s0
            fi
        fi

So we have a special case for c0t3d0s0. I assume this was necessary back when Sun hardware would present SCSI target 0 as sd3 (for SunOS 4) or c0t3d0s0 (for Solaris.) I'd actually forgotten about that little quirk until I ran across this snippet of code.

Copyright notice from the above-mentioned chkprobe script:

# Copyright 2005 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.

I was looking at the source code for devfsadm when I ran across something interesting in devfsadm.c:

    961  vprint(CHATTY_MID, "walking device tree\n");

I checked out devfsadm.h and found this:

     98 #define CHATTY_MID  "chatty"  /* prints with -V chatty */

I tried the "-V chatty" option and saw something a little more informative than a simple "failed to attach" message":

devfsadm[5355]: chatty: process_devinfo_tree: enter
devfsadm[5355]: chatty: lock_dev(): entered
devfsadm[5355]: chatty: mkdirp(/dev, 0x1ed)
devfsadm[5355]: chatty: process_devinfo_tree: attaching driver (rtls)
devfsadm[5355]: chatty: attempting pre-cleanup
devfsadm[5355]: chatty: devi_tree_walk: root=/, minor=, driver=rtls, error=0, flags=68
devfsadm: driver failed to attach: rtls
exit status = 1

(And of course, my first thought is, "Hey, a good starting point for DTrace!" Unfortunately, devfsadm is distributed as a stripped binary:

# ./devfsadm.d -c "devfsadm -i rtls"
dtrace: failed to compile script ./devfsadm.d: line 5: probe description pid5937::devi_tree_walk:entry does not match any probes
# file `which devfsadm`
/usr/sbin/devfsadm: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped
#

Guh. Not that it's the end of the world, but guh.

I was trying to use DTrace to watch the flow of control within a specific function in an application, and I saw some weird behavior. Specifically, what I was seeing was this:

  4        -> memmove
  4        <- memmove
  4      <- memcpy
  4      -> memmove
  4      <- memmove
  4    <- memcpy

After this, the indenting was out of alignment, tending more towards the left side of the screen than it should. Very annoying, but I was curious to see why I always seems to be returning from both memmove() and memcpy() when I should only have been returning from memmove(). (I wasn't seeing any entry points to memcpy() to match the returns.)

I went looking through the code at opensolaris.org, and here's what I saw:

     85  ENTRY(memmove)  /* (void *s1, void *s2, size_t n) */
     86  cmpq %rsi,%rdi / if (source addr > dest addr)
     87  leaq -1(%rsi,%rdx),%r9
     88  jle .CopyRight /
     89  cmpq %r9,%rdi
     90  jle .CopyLeft
     91  jmp .CopyRight
     92 
     93  ENTRY(memcpy)                        /* (void *, const void*, size_t) */
     94 
     95 .CopyRight:
     96 LABEL(1try):
     97         cmp     $16, %rdx
     98         mov     %rdi, %rax
     99         jae     LABEL(1after)
    100 
    101         .p2align 4

Hmm, so the two functions share the same code, or at least part of it. (The CopyLeft and CopyRight functions are copy in different directions. memmove() has to handle overlapping segments differently so that it doesn't overwrite memory that it hasn't moved yet, thus CopyLeft. memcpy() doesn't have this restriction, so it uses the simpler CopyRight.)

Given that these two share the same code in this fashion, they must necessarily share the same exit points, at least for code paths going through CopyRight. And that's why I was seeing two return probes firing, one for memmove() and another for memcpy(). In specifying all function entry and exit points, I'm causing DTrace to instrument both memmove() and memcpy(), and DTrace is doing this in such a way that there are two probes at this same point. When the executing process hits that point, it fires both probes, and I see a return from two different functions.

For the curious, here's the DTrace script I was running.

#!/usr/sbin/dtrace -s

#pragma D option flowindent

pid$target:dataserver:tdsrecv_language:entry
{
        self->traceme = 1;
}

pid$target:::entry,
pid$target:::return
/self->traceme/
{
}

pid$target:dataserver:tdsrecv_language:return
{
        self->traceme = 0;
        exit(0);
}