OmniOS Community Edition r151022en, r151032p, r151030ap OmniOS Community Edition

OmniOS Community Edition weekly releases for w/c 17th of February 2020 are now available.

This update requires a reboot for r30 and r32.

For all supported OmniOS releases, a sudo vulnerability (CVE-2019-18634 ) has been fixed.

For r30 and r32 additionally:

  • Fix for divide-by-zero bug in the i40e network driver illumos 9601

  • Update pkg to better handle packages with broken metadata

For r32 only:

  • PKCS#11 CKM_AES_CBC_PAD decryption could fail illumos 11825

  • Fix problem with the hal-storage-mount service where it would fail to mount things like Virtualbox guest additions

For further details, please see

Any problems or questions, please get in touch.

Goodbye Joyent Z IN ASCII - Writing

Goodbye Joyent.

FMA and Email Notifications Rob Johnston

[NOTE: This entry was originally posted on my blog on November 30th, 2010.  I have republished it here, as the information contained remains relavant to SmartOS and other Illumos distros]

In November, Oracle released a sneak peek at the next major release of Solaris in the form of Oracle Solaris 11 Express 2010.11.  There are tons of great features and innovations in this release.  One of the features I worked on was a new service smtp-notify, that can be configured to send email notifications in response to various Fault Management events, such as when a hardware component has been diagnosed as faulty.  Notifications can be configured for the following FMA event types (the descriptions below have been excerpted from the smf(1m) man page)


         A new problem has been diagnosed by the FMA   subsystem.
         The  diagnosis  includes a list of one or more suspects,
         which (where appropriate) may  have  been  automatically
         isolated  to prevent further errors occurring. The prob-
         lem is identified by a UUID  in the event  payload,  and
         further  events  describing  the resolution lifecycle of
         this problem quote a matching UUID.


         One or more of the suspect resources in a problem  diag-
         nosis  has  been repaired, replaced or acquitted (or has
         been faulted again), but  there  remains  at  least  one
         faulted  resource  in  the  list.  A repair could be the
         result of an fmadm command line  (fmadm repaired,  fmadm
         acquit,  fmadm  replaced)  or  might  have been detected
         automatically such as through detection of a part serial
         number change.


         All of the suspect resources in a problem diagnosis have
         been repaired, resolved or acquitted. Some or all of the
         resources might still be isolated at this stage.


         All of the suspect resources in a problem diagnosis have
         been  repaired  resolved or acquitted and are  no longer
         isolated (for example, a cpu that was a suspect and off-
         lined  is  now back online again; this un-isolate action
         is usually automatic).

If the smtp-notify package is installed, then the smtp-notify service is enabled out-of-the-box.

# svcs smtp-notify
STATE          STIME    FMRI
online         Oct_28   svc:/system/fm/smtp-notify:default

You can list the default notification preferences with svcs(1m):

# svcs -n
Notification parameters for FMA Events
    Event: problem-diagnosed
        Notification Type: smtp
            Active: true
            reply-to: root@localhost
            to: root@localhost

        Notification Type: snmp
            Active: true

        Notification Type: syslog
            Active: true

    Event: problem-repaired
        Notification Type: snmp
            Active: true

    Event: problem-resolved
        Notification Type: snmp
            Active: true

What does the above output tell us?  It tells us that problem-diagnosed events will result in an email notification being sent to root@localhost.  It will also result in a message being sent to syslog and an SNMP trap being generated.  Additionally, SNMP traps will be generated for problem-repaired and problem-resolved events.

What does an example email notification look like?  See below:

From Wed Jul 21 19:58:29 2010
Date: Wed, 21 Jul 2010 19:58:29 -0700 (PDT)
From: No Access User 
To: root@localhost
X-FMEV-CLASS: list.suspect
X-FMEV-UUID: e82aa706-ce6a-cbbb-a529-ceef1c9b57b0
Subject: Fault Management Event: diffuser:AMD-8000-AV

SUNW-MSG-ID: AMD-8000-AV, TYPE: Fault, VER: 1, SEVERITY: Major
EVENT-TIME: Wed Jul 21 19:58:29 PDT 2010
PLATFORM: Sun-Fire-X4200-Server, CSN: 0000000000, HOSTNAME: diffuser
SOURCE: eft, REV: 1.16
EVENT-ID: e82aa706-ce6a-cbbb-a529-ceef1c9b57b0
DESC: The number of errors associated with this CPU has exceeded acceptable levels.  Refer to for more information.
AUTO-RESPONSE: An attempt will be made to remove this CPU from service.
IMPACT: Performance of this system may be affected.
REC-ACTION: Schedule a repair procedure to replace the affected CPU.  Use 'fmadm faulty' to identify the module.

Those who've seen the messages that are logged to the console when FMA diagnoses a fault will see that the format is similar.  One additional thing to note is that each FMA email notification message also includes the following X-headers, which are there to aid admins who write mail filters:

Header Name Description
X-FMEV-HOSTNAME the name of the host on which the event occurred
X-FMEV-CLASS the event class
X-FMEV-CODE the Knowledge Article message ID
X-FMEV-SEVERITY the severity of the event
X-FMEV-UUID the UUID associated with the event

Email notification for FMA are highly configurable via svccfg(1m).  For example, you can enable/disable them per event type.  For example:

# svccfg setnotify problem-diagnosed mailto:active


# svccfg setnotify problem-diagnosed mailto:inactive

You can configure separate lists of one or more email recipients per event type.  For example:

# svccfg setnotify problem-repaired \,

You can even define your own message body template:

# svccfg setnotify problem-diagnosed \

Of course defining your own message template is nice, but it's only really useful if you have a way of referencing information about the actual FMA event in your message.  To facilitate this, we support the following expansion macros that can be embedded in message templates:

Macro Description
%% expands to a literal % character
%<HOSTNAME> expands to the hostname on which the event occurred
%<URL> expands to the URL of the knowledge article associated with this event
%<CLASS> expands to the event class
%<UUID> expands to the UUID of the event
%<CODE> expands to the knowledge article message ID
%<SEVERITY> expands to the severity of the event

But wait…there's more!

The smtp-notify service can also be configured to generate notifications for SMF service state transitions.  I won't go into the details of that here, but it's all documented in the smf(1m), svccfg(1m) and smtp-notify(1m) man pages.

A Sensor Abstraction Layer for FMA Rob Johnston

[Note: This entry was originally published on my site at and later  About 4 years ago Oracle deactivated many of the engineering blog sites (mine included). This was especially unfortunate as these blogs contained a wealth of technical content and in some cases provided the only existing documentation for certain Solaris and Illumos subsystems.  With the help of the waybackmachine, I have recovered my original blog content.  I am re-publishing two of my orignal entries, as they are the ones that still remain relevant today in that they document Solaris subsystems that were opensourced and live on in SmartOS and other Illumos distros today.  This first entry was originally published on August 6th 2008 - I have editted and added content and references in order to improve its relevance to Illumos]

Solaris Nevada (aka Solaris 11) build 96 is an important milestone build for the Sensor Abstraction Layer project for FMA, as it introduces the software infrastructure (the plumbing) on which the functionality described in the original design doc[1] will built.

This was a collaborative effort between the Solaris FMA team and the Fishworks team (specifically Eric Schrock) and involved over 7000 lines of change to over 60 source files.  Below are the two putback notifications that comprised the combined work:

First my putback done on July 31st:

PSARC 2008/428 Extending libnvpair for type double
PSARC 2008/463 Extending HC FMRI scheme to represent sensors/indicators
6579615 fmtopo -e has lots of memory leaks
6635159 libtopo: extend hc scheme to allow for representing sensors and indicators in the topology
6692392 fmtopo -x doesn't handle property methods properly
6718703 Need to extend libnvpair to support type double
6718712 libtopo: Need to implement facility provider module for IPMI
6722594 libtopo: the topo_prop_set_* interfaces need to learn to play well with propmethods
6727190 libtopo: add support for node properties of type double
6727459 libipmi: need interface to convert raw sensor readings to unit-based values
6727470 libipmi: need convenience routine to convert sensor unit defines to string
6729595 libtopo: add  case in fan and psu xml maps for SUN-FIRE-X4600-M2
6732318 fmd: small leak in sysevent modelling code

update: usr/src/cmd/fm/fmd/common/fmd_sysevent.c
update: usr/src/cmd/fm/fmtopo/common/fmtopo.c
update: usr/src/common/nvpair/nvpair.c
update: usr/src/lib/fm/topo/libtopo/
update: usr/src/lib/fm/topo/libtopo/common/hc.c
update: usr/src/lib/fm/topo/libtopo/common/libtopo.h
update: usr/src/lib/fm/topo/libtopo/common/mapfile-vers
update: usr/src/lib/fm/topo/libtopo/common/topo_2xml.c
update: usr/src/lib/fm/topo/libtopo/common/topo_error.h
update: usr/src/lib/fm/topo/libtopo/common/topo_fmri.c
update: usr/src/lib/fm/topo/libtopo/common/topo_method.c
update: usr/src/lib/fm/topo/libtopo/common/topo_method.h
update: usr/src/lib/fm/topo/libtopo/common/topo_mod.h
update: usr/src/lib/fm/topo/libtopo/common/topo_node.c
update: usr/src/lib/fm/topo/libtopo/common/topo_parse.h
update: usr/src/lib/fm/topo/libtopo/common/topo_prop.c
update: usr/src/lib/fm/topo/libtopo/common/topo_subr.c
update: usr/src/lib/fm/topo/libtopo/common/topo_subr.h
update: usr/src/lib/fm/topo/libtopo/common/topo_xml.c
update: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4500/Sun-Fire-X4500-hc-topology.xmlgen
update: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4540/Sun-Fire-X4540-hc-topology.xmlgen
update: usr/src/lib/fm/topo/maps/common/topology.dtd.1
update: usr/src/lib/fm/topo/maps/i86pc/chip-hc-topology.xml
update: usr/src/lib/fm/topo/maps/i86pc/fan-hc-topology.xmlgen
update: usr/src/lib/fm/topo/maps/i86pc/i86pc-hc-topology.xml
update: usr/src/lib/fm/topo/maps/i86pc/psu-hc-topology.xml
update: usr/src/lib/fm/topo/modules/common/Makefile
update: usr/src/lib/libipmi/
update: usr/src/lib/libipmi/common/ipmi_impl.h
update: usr/src/lib/libipmi/common/ipmi_sdr.c
update: usr/src/lib/libipmi/common/ipmi_util.c
update: usr/src/lib/libipmi/common/libipmi.h
update: usr/src/lib/libipmi/common/mapfile-vers
update: usr/src/lib/libipmi/common/
update: usr/src/lib/libnvpair/libnvpair.c
update: usr/src/lib/libnvpair/mapfile-vers
update: usr/src/pkgdefs/SUNWfmd/prototype_com
update: usr/src/uts/common/sys/fm/protocol.h
update: usr/src/uts/common/sys/nvpair.h
create: usr/src/lib/fm/topo/modules/common/fac_prov_ipmi/Makefile
create: usr/src/lib/fm/topo/modules/common/fac_prov_ipmi/fac_prov_ipmi.c

Examined files: 41

Contents Summary:
       2   create
      39   update

Names Summary:
       2   update parent's name history
       2   update children's name history

And now Eric Schrock's putback, done on August 1st:

PSARC 2008/485 SES Sensors and Enumerator
6720433 SES enumerator should provide controller revision information
6720435 SES enumerator should prefer description over class-description
6720452 SES enumerator should support indicators and sensors
6722807 SES enumerator should work with internal enclosures
6722809 want a way to identify enclosures as internal
6722811 SES enumerator should prefer elements with known status
6723603 x86 xmlgen topo scripts should make use of propmap
6732875 typo in fan-hc-topology.xmlgen
6732879 broken logic in pad_process()

update: usr/src/lib/fm/topo/libtopo/common/topo_parse.h
update: usr/src/lib/fm/topo/libtopo/common/topo_xml.c
update: usr/src/lib/fm/topo/maps/
update: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4200-M2/Makefile
update: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4200-Server/Makefile
update: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4500/Makefile
update: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4540/Makefile
update: usr/src/lib/fm/topo/maps/common/topology.dtd.1
update: usr/src/lib/fm/topo/maps/i86pc/fan-hc-topology.xmlgen
update: usr/src/lib/fm/topo/maps/i86pc/i86pc-hc-topology.xml
update: usr/src/lib/fm/topo/modules/common/ses/Makefile
update: usr/src/lib/fm/topo/modules/common/ses/ses.c
update: usr/src/lib/scsi/plugins/ses/Makefile
update: usr/src/lib/scsi/plugins/ses/libses/common/libses.h
update: usr/src/pkgdefs/SUNWfmd/prototype_i386
update: usr/src/pkgdefs/SUNWscsip/prototype_com
update: usr/src/pkgdefs/SUNWscsip/prototype_i386
update: usr/src/pkgdefs/SUNWscsip/prototype_sparc
update: usr/src/tools/scripts/
update: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4200-M2/Sun-Fire-X4200-M2-disk-hc-topology.xmlgen
rename from: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4200-M2/Sun-Fire-X4200-M2-hc-topology.xmlgen
         to: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4200-M2/Sun-Fire-X4200-M2-disk-hc-topology.xmlgen
update: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4200-Server/Sun-Fire-X4200-Server-disk-hc-topology.xmlgen
rename from: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4200-Server/Sun-Fire-X4200-Server-hc-topology.xmlgen
         to: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4200-Server/Sun-Fire-X4200-Server-disk-hc-topology.xmlgen
update: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4500/Sun-Fire-X4500-disk-hc-topology.xmlgen
rename from: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4500/Sun-Fire-X4500-hc-topology.xmlgen
         to: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4500/Sun-Fire-X4500-disk-hc-topology.xmlgen
update: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4540/Sun-Fire-X4540-disk-hc-topology.xmlgen
rename from: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4540/Sun-Fire-X4540-hc-topology.xmlgen
         to: usr/src/lib/fm/topo/maps/SUNW,Sun-Fire-X4540/Sun-Fire-X4540-disk-hc-topology.xmlgen
create: usr/src/lib/fm/topo/maps/common/xmlgen-header.xml
create: usr/src/lib/fm/topo/modules/common/ses/ses.h
create: usr/src/lib/fm/topo/modules/common/ses/ses_facility.c
create: usr/src/lib/scsi/plugins/ses/LSILOGIC-SASX28-A.0/Makefile
create: usr/src/lib/scsi/plugins/ses/LSILOGIC-SASX28-A.0/
create: usr/src/lib/scsi/plugins/ses/LSILOGIC-SASX28-A.0/amd64/Makefile
create: usr/src/lib/scsi/plugins/ses/LSILOGIC-SASX28-A.0/common/lsilogic.c
create: usr/src/lib/scsi/plugins/ses/LSILOGIC-SASX28-A.0/i386/Makefile
create: usr/src/lib/scsi/plugins/ses/LSILOGIC-SASX28-A.0/sparc/Makefile
create: usr/src/lib/scsi/plugins/ses/LSILOGIC-SASX28-A.0/sparcv9/Makefile

Examined files: 33

Contents Summary:
      10   create
      23   update

Names Summary:
       4   renamed
      10   update parent's name history
      14   update children's name history

First some background...

The Solaris Fault Manager maintains a snapshot of the hardware topology in a tree-like structure that includes a node for all hardware resources and FRU's that are managed/monitored by FMA.  The interfaces for generating a topology snapshot, walking the resulting tree and for manipulating the individual nodes in the tree are provided by libtopo and documented in Chapter 9 of the Fault Manager Programmer's Reference Guide.

The Sensor Abstraction Layer for FMA extends libtopo so that sensors and indicators can also be represented in our topology in a fashion that allows for the association of sensors and indicators to hardware resources to be programmatically determined.

Additionally, it introduces the concept of a facility provider module which provides an abstraction layer between libtopo and the lower-level interfaces that are used to control a given sensor or indicator.

Together this provides a set of common infrastructure to enable future FMA projects to manipulate sensors and indicators as part of Fault Management activities.

Below is some more details on three of the key new FMA infrastructure changes: hc FMRI scheme extensions, facility nodes and facility providers.

hc FMRI scheme extensions

Th existing hc-scheme allows for a heirarchial representation of hardware resources, according to their physical connection properties. However, this is not a very useful way to represent sensors and indicators in the topology because it does not allow for consumers to programmatically determine the association of sensors/indicators to the hardware resource that they're monitoring.

In Solaris Nevada build 96, we've extended the hc FMRI scheme to allow for this association to be represented using a new type of node in the topology: a facility node[1]

A facility node is a special leaf node in the hc-scheme topology that represents either a sensor or an indicator. A fault managed resource may have one or more child facility nodes that represent sensors or indicators that are associated with it. The hc-scheme was be extended as shown below to allow for an additional facility node member:

Name Data Type Description
scheme uint32 scheme used for FMRI
version uint32 version of scheme specification
authority nvlist optional authority of FMRI
payload resource path
facility nvlist facility component of FMRI

The facility nvlist will have two members:

Name Data Type Description
facility-type string type of facility node: "sensor" or "indicator"
facility-name string name of the facility (arbintrary)

The string representation of an hc scheme FMRI will also be extended, as shown below:


for example:



Facility Nodes

As mentioned, facility nodes can represent either sensors or indicators.

An indicator is intended to be an abstraction representing one or more LEDs on the hardware component. Three types of indicators are supported: locate, fault and ok2rm

In some cases a given indicator will in fact correspond directly to a dedicated physical LED. In other cases, a single LED may be overloaded to support multiple types of indicators. For example, disk drive bays often do not have separate locate and fault LEDs. Instead, a single LED may be used with a solid-on or blinking state indicating either fault or locate, respectively. Similarly, a single indictor could correspond to multiple physical LEDs. For example, setting the chassis fault indicator to ON may result in amber LEDs being illuminated on both the front and back of the chassis.

Sensors are further broken down into two to subtypes:  threshold and discrete.

Threshold sensors represent sensors that provide both an analog sensor reading as well as a discrete state value (indicating whether certain upper or lower thresholds have been crossed).  Possible examples of a threshold sensor are ambient temperature sensors or a fan tachnometers.

Discrete sensors only provide a discrete state value - for example, a chassis intrusion sensor.

Facility Providers

A facility provider is a logical collection of node and property methods that provide an abstraction layer between libtopo and the underlying lower-level interfaces that are used to actually manipulate the sensors and indicators. This allows library consumers (namely fmd) to access sensor readings and manipulate indicators via standard libtopo interfaces (e.g. topo_prop_{get|set}_{type}). Nevada build 96 includes the implementation of facility provider modules for IPMI[2] and SES[3], which will provide broad coverage across Sun's x64 server platforms.

Facility providers are implemented as simplified libtopo plugin modules (similar to enumerator modules). However, in the implementation of their tmo_enum entry point, a facility provider will simply register its methods on the node that is passed in.

[Note: Illumos has been further enahnced with provider modules that support AHCI enclosure services,  drive indicators for direct-attached SAS drive bays when using Broadcom/LSI SAS adapters and sensors X86 chip and platform sensors. (see Robert Mustacci's blog about the latter here).  Future work in Illumos will add a provider for controlling indicators on NVMe/U.2. form-factor drive bays]

Facility nodes are intended to be self-describing and as such are required to have certain properties set in the "facility" property group.  Facility provider modules provide a set of methods for discovering sensors and getting/setting a set of required properties (described below)

Facility Node Properties

Property Name Property Type Threshold Sensors Discrete Sensors Indicators
type uint32 required required required
sensor-class uint32 required required N/A
reading double required N/A N/A
state uint32 required required N/A
units uint32 required N/A N/A
mode uint32 N/A N/A required
threshold-lower-non-critical double optional N/A N/A
threshold-lower-critical double optional N/A N/A
threshold-lower-non-recoverable double optional N/A N/A
threshold-upper-non-critical double optional N/A N/A
threshold-upper-critical double optional N/A N/A
threshold-upper-non-recoverable double optional N/A N/A

[NOTE: The upper/lower threshold properties were later added in Illumos via my commit here]

Libtopo provides a rich set of #defines that are used for setting the required and optional facility node properties.  As an example:

The units should be set to one of the following defines:

typedef enum topo_sensor_unit {






} topo_sensor_unit_t;

Much more information is available in the following header file


Below are some example excerpts of fmtopo[4] output for both a threshold and discrete sensor as well as an indicator node. These examples were taken from a Sun-Fire X4500.

  group: facility                       version: 1   stability: Private/Private
    entity_ref        string    proc.p0.t_core
    sensor-class      string    threshold
    type              uint32    0x101 (THRESHOLD_STATE)
    state             uint32    0x0 (0x00)
    reading           double    49.000000
    units             uint32    0x1 (DEGREES_C)

  group: facility                       version: 1   stability: Private/Private
    entity_ref        string    ps0.prsnt
    sensor-class      string    discrete
    type              uint32    0x108 (GENERIC_PRESENCE)
    state             uint32    0x2 (ASSERTED)

  group: facility                       version: 1   stability: Private/Private
    entity_ref        string    hdd40.state
    mode              uint32    0x1 (ON)
    type              uint32    0x3 (PRESENT)

[1] The hc FMRI scheme extensions and the concept of facility nodes are based on Cynthia McGuire's original design document for the Sensor Abstraction Layer, so readers may find it beneficial to review section 2.3 of that document to gain additional background.

[2] IPMI is an acronym for Intelligent Platform Management Interface which is an Intel specification for doing out-of-band management of computers.  It has become and industry standard and thus most x86 server platforms contain BMCs that support IPMI.  Through IPMI we can get access to platforms sensors and (sometimes) indicators.

[3] SES is an acronym for SCSI Enclosure Services which is a protocol for accessing diagnostic services for SCSI storage enclosures including things like temperature and voltage sensors.   SES also provides mechanisms for toggling the state of drive bay and chassis LEDs on storage enclosures.

[4]  fmtopo is a command-line utility that developers can use to take a snapshot and dump the resulting topology.  It's usage is documented in chapter 12 of the Fault Manager Programmer's Reference Guide.

One big Oracle ASM adventure alp's notes

Well, this has happened long ago, but as I'm going to decomission, I think this information should be moved here...

Originally posted on on Tue, 04/07/2009

I had very unpleasant experience with RAC and ASM. I'd like to share it. Yesterday after VMware ESX Server upgrade (and maybe unsuccessfull live motion operations on one of two RAC nodes) I had the following records in asm instance alert log:

Mon Apr 6 15:48:52 2009
NOTE: cache registered group DATA number=1 incarn=0x3ad84292
NOTE: cache registered group FRA number=2 incarn=0x3b084293
NOTE: cache registered group LOGS number=3 incarn=0x3b084294
Mon Apr 6 15:48:52 2009
ERROR: no PST quorum in group 1: required 2, found 0
Mon Apr 6 15:48:52 2009
NOTE: cache dismounting group 1/0x3AD84292 (DATA)
NOTE: dbwr not being msg'd to dismount
ERROR: diskgroup DATA was not mounted
Mon Apr 6 15:48:52 2009
ERROR: no PST quorum in group 2: required 2, found 0
Mon Apr 6 15:48:52 2009
NOTE: cache dismounting group 2/0x3B084293 (FRA)
NOTE: dbwr not being msg'd to dismount
ERROR: diskgroup FRA was not mounted
Mon Apr 6 15:48:52 2009
ERROR: no PST quorum in group 3: required 2, found 0
Mon Apr 6 15:48:52 2009
NOTE: cache dismounting group 3/0x3B084294 (LOGS)
NOTE: dbwr not being msg'd to dismount
ERROR: diskgroup LOGS was not mounted

Diskgroups disappeared, and asm didn't want to see them. However, I could see /dev/sd* disks, where diskgroups were placed, and permissions were right: disks were owned by oracle user. select * from v$asm_diskgroup ; gave nothing and select path,header_status from v$asm_disk ; said that all disks were in provisioned state. It was awful, but in the end I've found sollution. I've noticed that kfed sees diskgroup and other attribute on the affected disks, but parameter kfdhdb.acdb.ub2spare is not 0 (as I found in Internet, it was its usual state). So I've dumped header info:

$ kfed read /dev/sdf > /tmp/sdf.noop.mod
$ vi /tmp/sdf.noop.mod #Changed kfdhdb.acdb.ub2spare to 0
$ kfed op=write dev=/dev/sdf text=/tmp/sdf.noop.mod CHKSUM=YES
After this I could do "alter diskgroup fra mount" in asm instance, repeated this procedure for DATA asm diskgroup, after that I could mount diskgroups and recover my database. In conclusion I wish say that we are going to move our fra to OCFS2 fs the next weekend.

What is my sftp server doing? alp's notes

Well, I'm not familiar with DTrace, but sometimes want to find, what some application is doing. In this case I wanted to monitor my sftp server. Luckily, most illumos distributions provide dtrace patch (coming from Oracle Solaris) to find this out. Unluckily, I haven't found any documentation on it, just source code. After reading Translators chapter of DTrace Guide and looking at /usr/lib/dtrace/sftp.d I've come to this:

dtrace -n 'sftp*:::transfer-done { printf ("%d: %s %s %s %d", pid, xlate <sftpinfo_t *>((sftpproto_t*)arg0)->sfi_pathname, xlate <sftpinfo_t *>((sftpproto_t*)arg0)->sfi_user, xlate <sftpinfo_t *>((sftpproto_t*)arg0)->sfi_operation, xlate <sftpinfo_t *>((sftpproto_t*)arg0)->sfi_nbytes ); }'

dtrace: description 'sftp*:::transfer-done ' matched 8 probes
1 80412 process_read:transfer-done 7409: /export/home/user/1.pp user read 1808
1 80412 process_read:transfer-done 7409: /export/home/user/1.pp user read 0
1 80411 process_write:transfer-done 7409: /export/home/user/1.pp user write 1808
1 80412 process_read:transfer-done 7409: /export/home/user/dtrace/poll.d user read 53
1 80412 process_read:transfer-done 7409: /export/home/user/dtrace/poll.d user read 53

Seems rather interesting to me.

Quest: creating one hundred zones alp's notes

Well, I need to create about one hundred zones once again. You could probably use ansible for this, but an old-fashioned man will do everything in shell. So: we have one "golden image" and have to create 100 zones like it. We could clone it, but with clones you receive wonderful issue - beadm activate fails in zone. So we create zones and do send/receive manually. This looks like this:

set -e

for i in $(seq 1 100); do

#Creating interface for the zone
dladm create-vnic -l e1000g1 hnet$i

#Creating initial config

create -b
set zonepath=/zones/h$i
set autoboot=true
set ip-type=exclusive
add net
set physical=hnet$i
add capped-memory
set physical=2G
add rctl
set name=zone.max-swap
add value (priv=privileged,limit=2147483648,action=deny)
add rctl
set name=zone.max-locked-memory
add value (priv=privileged,limit=536870912,action=deny)

zonecfg -z h$i -f $TEMPFILE
zfs send -R data/zones/h0@initial | zfs recv -F data/zones/h$i

# Zone tools should know that zone is in installed state, not configured
# Also during installation zoneadm assigns uuid to zone (last field). We do this manually.
gsed -i -e "/^h${i}:/ s/\$/${uuid}/" -e "/^h${i}:/ s/configured/installed/" /etc/zones/index
zoneadm -z h$i mount

# We known that golden image ip address ends in 254 and change it
sed -i -e "s:hnet0:hnet$i:g" -e "s:\.254:.$addr:g" /zones/h$i/root/etc/ipadm/ipadm.conf
zoneadm -z h$i unmount
zfs destroy data/zones/h$i@initial
zoneadm -z h$i boot

A brief story of how you shouldn't promote your open source project alp's notes

I'll just leave it here . And will block any attempt to integrate Pale Moon in our repository. Just to protect our developers from such attitude and trolling.

Operating system materials alp's notes

Does ip belong to network? alp's notes

It's so easy to check if IP belong to network... Until you start doing this in shell. I've tried and finally got this. This version works in bash, dash and ksh... Good enough for me, but perhaps it could be optimized a bit to avoid cut usage. Our function gets two parameters - ip address and network in address/netmask format. In fact we compare IPaddress & netmask and IPnetwork & netmask.


belongs_network ()

netaddr=`echo $network | cut -d / -f 1`
netcdr=`echo $network | cut -d / -f 2`

a1=$(echo "$addr" | cut -d . -f 1)
a2=$(echo "$addr" | cut -d . -f 2)
a3=$(echo "$addr" | cut -d . -f 3)
a4=$(echo "$addr" | cut -d . -f 4)

n1=$(echo "$netaddr" | cut -d . -f 1)
n2=$(echo "$netaddr" | cut -d . -f 2)
n3=$(echo "$netaddr" | cut -d . -f 3)
n4=$(echo "$netaddr" | cut -d . -f 4)


if [ $ares -eq $nres ] ; then
return 0
return 1

if belongs_network; then
echo "belongs"
echo "does not belong"

Thank you, Oracle engineers alp's notes

After 2010, when Oracle acquired Sun, most of us, who followed OpenSolaris, were depressed. In one year one of the most advantageous operating systems was closed under steel curtain. Luckily, due to enormous efforts of community, of companies, dependent on OpenSolaris, the system survived. Currently we have several more or less successful illumos distributions, targeting different users. But nowadays there's a (of course, deserved) common negative feeling towards Oracle in illumos community. But let's speak from another point of view. Let's look at things, which illumos community (and in particular, OpenIndiana) got directly or indirectly from Oracle in recent years.

  • Our userland build system, which constantly evolves, however, in different directions, under Oracle control and in our distribution. But still a lot of components can be easily migrated between build systems.
  • A lot of software build receipts and patches, as result, were borrowed with small modifications, from Oracle userland-gate. The process is still going on.
  • We still borrow patches from Solaris pkg-gate. Also differences in underlying kernels are currently rather significant, a lot of changesets from pkg-gate can be ported to OpenIndiana pkg5 repository.
  • Of course, I can not avoid thanking Alan for his constant help in supporting Xorg subsystem and GUI parts of our distribution. He was always helpful to me and Aurélien.
  • Evidently, recent KMS work, integrated into OpenIndiana, wouldn't be possible without Oracle's open drm port, which was ported from Solaris to illumos by Martin Bochnig, and later independently ported and enhanced by Gordon Ross.
- And of course, I cannot count patches, which were suggested to upstream projects by Oracle engineers. Just today when I tried to solve two issues related with IPS and apache 2.4 interaction, I've found two patches by Petr Sumbera, fixing Apache issues on Solaris. So, I want to use the chance and thank all Oracle Solaris engineers for their work on open source projects. I doubt that without them illumos could survive in large scale. Perhaps, we could be an excellent playground for ZFS development, but not an universal operating system...

Enterprise Information systems alp's notes

This year I'm going to read "Enterprise Information systems" course for students of Southern Federal University Institute for Advanced Technologies and Piezotechnics. The materials of this course will be appearring in this Microsoft Class Notebook Google docs link

Sending SMS notifications from OpenIndiana zone with Huawei E1550 alp's notes

I want to receive SMS notifications when something is terribly wrong in our data center. One part of the problem is OpenNMS, and we know how to configure it. But another part is actually sending SMS.

As only physical server with normal USB ports which we have is self-assembled SuperMicro storage, running OpenIndiana, we want to create OI zone, which can send SMS. First of all, we'll need some USB modem. I use Huawei e1550, which was already switched to modem-only mode (AT^U2DIAG=0) on Windows PC. Let's look at prtconf -v (looking for HUAWEI):

device, instance #0
Driver properties:
name='pm-components' type=string items=3 dev=none
value='NAME= usbsacm0 Power' + '0=USB D3 State' + '3=USB D0 State'
Hardware properties:
name='driver-minor' type=int items=1
name='driver-major' type=int items=1
name='high-speed' type=boolean
name='configuration#' type=int items=1
name='usb-product-name' type=string items=1
value='HUAWEI Mobile'
name='usb-vendor-name' type=string items=1
value='HUAWEI Technology'
name='usb-raw-cfg-descriptors' type=byte items=85
name='usb-dev-descriptor' type=byte items=18
name='usb-release' type=int items=1
name='usb-num-configs' type=int items=1
name='usb-revision-id' type=int items=1
name='usb-product-id' type=int items=1
name='usb-vendor-id' type=int items=1
name='compatible' type=string items=3
value='usb12d1,1001.0' + 'usb12d1,1001' + 'usb,device'
name='reg' type=int items=1
name='assigned-address' type=int items=1

If modem was switched to modem-only mode, you'll see

value='usb12d1,1001.0' + 'usb12d1,1001' + 'usb,device'
in 'compatible' field.So, let's say OI that it should use usbsacm to talk to it:
# update_drv -a -i 'usb12d1,1001' usbsacm

Now you should have /dev/term/0 - 3 devices and can use tip to talk to /dev/term/0:

# tip /dev/term/0

Let's create zone and pass /dev/term/0 to the zone:

# zonecfg -z sms1
sms1: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:sms1> create
zonecfg:sms1> set zonepath=/zones/sms1
zonecfg:sms1> set brand=ipkg
zonecfg:sms1> set autoboot=true
zonecfg:sms1> set ip-type=exclusive
zonecfg:sms1> add net
zonecfg:sms1:net> set physical=sms10
zonecfg:sms1:net> end
zonecfg:sms1> add device
zonecfg:sms1:device> set match=/dev/term/0
zonecfg:sms1:device> end
zonecfg:sms1> exit
# zfs create data0/zones/sms1
# zoneadm -z sms1 install
# zoneadm -z sms1 boot

Now we can configure network in our zone:

# zlogin sms1
# ipadm create-if sms10
# ipadm create-addr -T static -alocal= sms10/v4
# route add -p -net 0
# cp /etc/nsswitch.dns /etc/nsswitch.conf
# echo 'nameserver' > /etc/resolv.conf
# svcadm enable dns/client

We'll use gammu utilities to send sms:

# pkg install utility/gammu

Your /etc/gammurc (or ~/.gammurc) should look like:

device = /dev/term/0
connection = at

If everything is fine, now you can send SMS:

# gammu sendsms TEXT +7myphone -text "test"

My Crimea trip (April-May 2016) alp's notes

This time I had little time to plan my holidays. So, I didn't have time to get Visa. After repairing my flat, I also didn't have a lot of money, and tried to find something interesting and cheap. Luckily, my travel agent advised me a trip to Crimea. I was frightened by long bus road, but plain tickets costed almost as all trip, so decided, why not and went to Alushta by bus. I tried to make some notes during the trip, they are presented later.

Day 1. The road

What can you expect from bus tour to Crimea for 170 euros? Everything is possible. In the beginning we found out that bus has opaque windows and malfunctioning microphone. Also one of the drivers forgot his documents, so we stayed in Azov for 2 hours waiting for another bus.  So we walked in Azov city park for some time. After leaving park we had to wait some more time on the road. So at last we left Azov at 7-8 in the evening.
When we finally left Azov,  driver turned on a film, which killed me with its stupidity. Luckily, the DVD disk was damaged, and we didn't have to watch all the film. There's no good light in the bus, so I can't read my book. But I can hear loud conversations of patriotic company which drink cognac on the backseat of the bus. So I'm listening to Frank on my phone and hope that it will not discharge too soon.
Ferry… Passenger control was surprisingly fast, but our bus registration took some time. So the overall process took about 2.5 hours. One interesting thing I've seen while waiting for the bus registration is boarding of railway oil tanks boarding the ferry.

Day 2. Alushta. Nikitsky botanical garden. Massandra palace

So, we still in the road. I hope I'll be able to sleep at the hotel. Vain hope. I've just managed to unpack my bag, take a shower and take a breakfast. And back to the bus.
First stop is Nikitsky botanical garden. The park is big and beautiful. But there's no any salt, which would impressed me (like Japan maples).  Perhaps, I'm just too sleepy.

After visiting the garden, we hurry to Massandra palace, a former dacha of Russian monarchs (and  recreation place for general secretaries). The palace looks really fine, however it would benefit from some restoration. Inside the palace everything is similar to other palaces in St. Petersburg or Vienna. I can't expect that a man can comfortably live there, it's too luxurious (and boring). 

Day 3. Livadia palace. Lower Oreanda Park

Ufff. It's cold. Terribly cold. I couldn't turn on conditioner in heating mode, so had to sleep  under 3 blankets. It's about 10 degrees outside. But under 3 blankets it's warm enough. Of course, there's no any central heating in this summer hotel. In the morning hotel staff explained me that air conditioner remote buttons should be pressed harder and helped to turn on air conditioner in heating mode.
In the morning we are going to Livadia palace. I liked it. It's very light, big and decorated with taste. As I understood, it was finished in the end of 19th century, and Nikolay II liked to spend his spare time there. During excursion we were told about this intelligent, almost saint, tsar… But he seems more like miserable tsar. Lost Russian territories, allowed country to fall apart and was killed by his own nation… Not a bright figure for an emperor.  Nikolay had the largest private collection of cars in Europe.. and especially high rate of illiteracy in the country. They say he liked to rest in Yalta, make photos. Kodak has presented him special camera to make panoramic photos. The photos of his family are exhibited in the palace.

In the palace there's also an exhibition devoted to the Yalta conference. The palace itself was given as residence for Roosevelt. There's a copy of "Pravda" newspaper of 1945 with printed decisions of the conference. You can see a sculpture of Churchill, Roosevelt and Stalin near sanatorium, which is situated nearby. It wasn't placed near the palace, as it's too heavy for this place.

There is a pathway which starts near the palace. Originally it was used by tsar's family. Now it leads to the Lower Oreanda Resort. This resort was used by  Central Committee of the CPSU members and especially honoured men. Almost every Russian actor, a lot of writers were here. Even Korolev visited this place. The pathway is really cool. It looks like similar pathways in Kislovodsk's parks, only surrounded with southern exotic plants.  I'd like to spend there more time, but as always we were in a  hurry. So we ascended to the arc in the hills, and then descended to the mentioned resort. The road down is rather steep, so it was interesting.
Near this resort there's a church. A priest who serves there is famous in Crimea. He writes theological and philosophical books. Also he is known, because he doesn't require payments for his wedding and memorial services.
After visiting Resort's park there was a concert of church choir music. As I barely could withstand this, I spent this time walking in the park, which looked lovely.
We were going back to the bus along the same pathway. The pathway is called "Sunny path" and to justify this name, sundial was placed near the path. However, it never shows true time and serves only as decorative element.
In the evening I forgot a packet with food in the bus, so I had to go walking, searching for a cafe. I've walked along the beach, but all cafes there were still closed or were not very attractive. Finally I've chosen one with adequate prices and interesting interior - guns, steering wheels, bear's skins. Food was delicious, but music was awful. As it was live music, they even included some margin for it in the bill. It wasn't high, but I'd prefer to pay it to these musicians for silence, not for their music.

Day 4. Vorontsov Palace. Swallow's Nest. Yalta.

After breakfast we were going to the Vorontsov Palace. In the road the guide talked us about count Vorontsov. It was one of the richest russians of his time. But unlike modern Russian thiefs oligarhs, he spent a lot of own money on the state business - he built roads and developed cities.  When after battle he lost most of his 9000 soldiers, he rebuilt one of his estates into the hospital and all of his remaining wounded soldiers were nursed there. When they recovered, he paid them significant rewards and gave everyone a fur coat (it was in winter). He has even sold another estate to cover the costs of carousing hussars in Paris  when our troops were leaving it after first Patriotic War...
In any case, his palace is great. It looks like a Medieval castle, and is made from semiprecious material. And of course, white lions are majestic. There's an English park beyond his palace, with pounds, swans and so on… It's evident that this palace was created by one of the richest man in Russian Empire. They say, Nikolay I was angry when saw this palace, as it was more beautiful than his own palaces. But the next morning he pardoned Vorontsov and asked him to oversee the building of Emperor palace nearby (Livadia palace). By the way, Vorontsov Palace was Churchill's residence during Yalta conference. It's not a big surprise, as this place resembles European castles.
After the palace we went to the Swallow's Nest. This castle was sold several times. His last owner before the Revolution was some merchant, who wanted to make a restaurant there. In Soviet era, the castle was in pity state. It needed reconstruction. Some works were made in the mid of the century, and it was transformed into museum. Several films were shot there, including "Ten Little Niggers". Ukrainian government leased this palace to Italian company, who tried opened restaurant. They didn't know the history. After several years Ukrainian president voided the contract and turned palace into museum again. He even repaired it for state's money. They say he wanted to privatize the palace, but didn't have time to finish with his plans, as Maidan has happened. Now it's a museum again.
From Swallow's Nest we went by small ship to Yalta. In Yalta we had some free time. I even managed to have a decent dinner there for ~ 2 euro. While walking along the seafront, I've seen several concerts - they were held on the stages under clear sky. There were a lot of publice there. Some of them tried to get their money, lending different exotic personal vehicles, forcing you to take a picture with their animals or playing violins. Lively place. I was fond of several pairs of old men, playing chess on the benches. A man managed to do check-mate in 6 moves from situation, which didn't foretell it in any way. 
In the evening I walked from the hotel in the direction opposite to the city center. Saw several pictures in Stalker style - abandoned children recreation centers, rusty satellite antennas, abandoned wagons…

Day 5. Genoese fortress. Vine plant.

At 7 o'clock we left for Sudak. During our way guide talked about Alans, Greeks, Genoeses, Tatars, Turks and other nationalities, living in Crimea in different times. She even read some poems. Churches and mosques were seen along our way. In 1945 a lot of tatars were deported from these lands, and all original titles were changed in one night. So town names like Perevalnoe, Svetloe, Pionerskoe appeared. The Tatars in awful conditions, in goods wagons, were transferred to Kazakhstan.
Sometimes we see monuments to partizans. They created a lot of troubles for fascists, but also often died. Germans burned and cut down forest, trying to secure roads for their transports, but everything was in vain.
First event for today is Genoese fortress. However, earlier it was Byzantine fortress, and before this it was Alanian fortress. Looks great. The spirit of the place a bit resembles Tanais ruins. I'd gladly walked here for half a day, but we have only one hour. Almost whole or well reconstructed towers, walls,  battlements. Hipster-style guide gives us brief information about different fortifications in the fortress. There is an interesting building in the fortress. It's a mosque, which was built by Turks, who one day came here. When they left, it became Catholic church. Later it was used as sinagoga by local Jews. This holy place now is a museum. You should appreciate that internal area of the fortress housed a small city. There were not enough Genoeses here to defend such big fortress, so during hard times either militia was used, or people have to live the fortress and the city and hide in the forest.
Next stop is vine cellars of "Solnechnaya Dolina" vine plant. The plant produces a lot of vines, and we are told how vine is made. They say, the cost of one oak barrel for vine is about 1000 euros. If it's just cost of empty barrel, how much will full barrel cost? The cellars were founded in the end of  XIX century by Golitsyn, but changed their owners rather often.  Local people managed to steal vine, using ventilation holes, so in middle of the century they were locked with bars.  The cellars successfully survived WWII, because one of the German high officers liked winemaking and wanted to grab the plant, but you know the history. Now plant produces a lot of vines, the main brands include "Black doctor" and "Black colonel". I wanted to get "Black doctor", as my friend liked his cheap replica, but it costed as 4-5 bottles of Abkhazian wines, so I stopped on portvein, which was cheaper.
As I forgot to buy any food during the day, I have to buy "Snickers" for the cost of the second course in the canteen where I usually eat. Another impressions on the ferry are related to the ferryboat itself. It looks grandiosely. It was interesting that ferryboat does two turns during its way so that it lands with its back part…


There were a lot of impressions, and most of them were positive. I saw that this region is developing with great speed. From several conversations I found out that people expect that a lot of funding will come from Moscow after 2018, when World Championship will be over and Kerch strait bridge will be finished. Of course, this bridge is a necessity, as crossing strait on the ferry takes too much time.
I still haven't seen a lot in Crimea, and I'd like to go back and visit Sevastopol and Bakhchysarai, to spend several days on the sea shore.
If you decide to go there, don't forget that for now it seems only MTS mobile services work there. Also several men in our group had issues with their credit cards, so don't forget to get cash. Luckily, you'll not need a lot of money there.

Converting "linked images" zones to non-linked alp's notes

A while ago we introduced "nlipkg" zone brand in OI to create "non-linked" images. OmniOS uses ipkg as non-linked brand by default and has additional "lipkg" brand for linked images. Briefly speaking, when you deal with linked images, global zone's IPS knows a lot about zones, can work with them (for example, you can update all zones in one step with "pkg update -r") and imposes some restrictions on child images. Zone's brand is recorded in /etc/zones/zonename.xml and can be changed manually or using zonecfg. As ipkg and nlipkg zones are rather similar (in fact, they are distinguished only in name and IPS checks in some places on which brand it's operating, but for these two brands zone brand scripts are the same). So, when you are bugged with IPS checks for linked images, you can try to change zone's brand from ipkg to nlipkg. This even can work. The only issue is that it doesn't always work. Sometimes you still receive irritating messages like

pkg install: Invalid child image publisher configuration. Child image publisher
configuration must be a superset of the parent image publisher configuration.
Please update the child publisher configuration to match the parent. If the
child image is a zone this can be done automatically by detaching and
attaching the zone.

The parent image has the following enabled publishers:
PUBLISHER 0: (non-sticky)
PUBLISHER 1: userland (non-sticky)
PUBLISHER 2: hipster-encumbered

The child image has the following enabled publishers:
PUBLISHER 0: (non-sticky)
PUBLISHER 1: hipster-encumbered
Even for nlipkg-branded zones. The issue is that inside zone IPS knows nothing about zone's brand. Its logic is always the same. The only thing which it checks for are files in /var/pkg/linked directory. These files are usually created on pkg operations initiated from GZ (the same pkg update -r) and contain information about parent image (read - GZ). When you change zone's brand, they will not disappear, and IPS inside zone will still think that it works with linked image. Luckily, to convince it that it's not true, it's enough to do "rm -fr /var/pkg/linked". Then this condition will make IPS happy. So, long story short - don't forget to remove /var/pkg/linked if you convert zone from ipkg to nlipkg brand.

Bye-bye, sysidtool, hello sysding alp's notes

Often you want to have some simple tool to configure basic system settings after installation, such as ip settings, time zone settings, locales and so on. More important, installer also sometimes needs similar utility, which would run on first boot and initialize basic system parameters. For example, OmniOS runs /.initialboot script on the first boot. Solaris historically had sysidtool service, which read /etc/sysidcfg file and used it to configure zones or base system. Sysidtool also had ncurses-based interface to perform basic zone configuration.
The disadvantage of sysidtool is that it is a closed source tool, and you cannot fix it if you want it to do a bit more. So, we switched to sysding in OpenIndiana Hipster.
Sysding was originally written by Olaf Bohlen (Agnar at #oi-dev) to configure multiple illumos/Solaris zones. It doesn't have interactive interface, but has a set of utility functions to write configuration scripts. File /etc/sysding.conf is a simple ksh script, sourced by /lib/svc/method/sysding on . /lib/svc/method/sysding predefines some useful functions which can be necessary for initial configuration. Sample configuration file can look like

setup_timezone Europe/Moscow
setup_locale en_US.UTF-8
setup_user_password root '$5$+Fu+utqXFqU=$RD2LbFipqwKc2srNFYnVkda9U6K2pmMajvuR3iyHzR'
setup_interface PRIMARY v4
setup_route default
setup_ns_dns "stud.lan" "stud.lan notebook.lan" ""

Using it, sysding will set timezone, locale, root password, network settings on first boot and reboot zone (or NGZ), because /etc/default/init was changed. It also cares about setting root password to 'NP' at first boot in zone if it's empty and you haven't specified one. Without this you wouldn't be able to "zlogin" to the zone. It can do a bit more. If you are interested, look at /lib/svc/method/sysding . If you want to have some customizations for your environment, create pull requests against, but don't forget two things: test your changes thoroughly and keep in mind that sysding was created to be a simple configuration tool.

Materials for Unix courses alp's notes (Yes, I needed some fast way to share link :))

Userland incorporation in OpenIndiana Hipster and what does it mean for developer alp's notes

Last Sunday we've published userland incorporation to /hipster-2015 repository. This was a feature long asked for by several users. It is generated by Jenkins on the build host and forces all packages generated by oi-userland build (excluding illumos-gate-provided packages and kvm) to be the latest. As we publish this incorporation on each oi-userland rebuild, you can force your system to go to specific point in the future. For example:

$ pkg list -avf pkg://
pkg:// ---
pkg:// ---
pkg:// i--
pkg:// ---
pkg:// ---
pkg:// ---
pkg:// ---

We can see that this system has incorporation with  20151004T163056Z  timestamp installed and two more recent versions are available. So I can do something like:
$ sudo pkg update -v \

to move to the 5th October  package versions.

Why developers doesn't like incorporations (and IPS generally)? Because it doesn't allow you to do what you want with your system. For example, you can't install another package version.
In case of userland incorporation, you have some freedom. You can just uninstall it (and entire, as entire depends on userland-incorporation). But you can get issues during new zone setup if your NGZ misses entire. So, you have three options:
  • stay with old entire, which doesn't depend on userland-incorporation (pkg freeze it);
  • uninstall entire and userland-incorporation;
  • use facets to relax incorporate dependencies.

Let's look on the last option more attentively. For example, I'm going to experiment with new mesa. But userland-incorporation has the following dependency:
$ pkg contents -m userland-incorporation |grep mesa
depend facet.version-lock.x11/library/mesa=true fmri=x11/library/mesa@10.5.9,5.11-2015.0.1.0:20150927T212600Z type=incorporate

As you see, it is marked by facet. So you can do
$ sudo pkg facet facet.version-lock.x11/library/mesa
version-lock.x11/library/mesa True system
$ sudo pkg change-facet facet.version-lock.x11/library/mesa=false
$ sudo pkg install pkg://userland/x11/library/mesa
$ sudo pkg info mesa
Name: x11/library/mesa
Summary: The Mesa 3-D Graphics Library
Category: System/X11
State: Installed
Publisher: userland
Version: 11.0.2
Branch: 2015.0.1.0
Packaging Date: October 7, 2015 03:00:58 PM
Last Install Time: October 7, 2015 06:49:24 PM
Size: 34.82 MB
FMRI: pkg://userland/x11/library/mesa@11.0.2-2015.0.1.0:20151007T150058Z
Project URL:
Source URL:
But beware, if you would like to change facet back, you can't do it:
$ sudo pkg change-facet version-lock.x11/library/mesa=true
Creating Plan (Solver setup): /
pkg change-facet: Package entire must be uninstalled before the requested operation can be performed.
Reject: pkg://
Reason: No version matching 'require' dependency consolidation/userland/userland-incorporation can be installed
Package x11/server/xorg/driver/xorg-video-ati must be uninstalled before the requested operation can be performed.
Reject: pkg://
Reason: No version matching 'require' dependency x11/server/xorg@1.14.7-2015.0.1.0 can be installed
Reject: pkg://
Reason: No version matching 'optional' dependency x11/library/mesa@7.4.4-2014.1.3.0 can be installed
Reject: pkg://userland/x11/library/mesa@11.0.2-2015.0.1.0:20151007T150058Z
Reason: This version is excluded by installed incorporation consolidation/userland/userland-incorporation@0.5.11-2015.0.2.0


Firstly, you have to install mesa version offered by publisher.
Update (2016-01-26). If you use your host as test station, it's easier just to uninstall userland-incorporation. Now you can do this without touching entire. Just do

$ sudo pkg change-facet facet.require.consolidation/userland/userland-incorporation=false
$ sudo pkg uninstall userland-incorporation

Don't think for me... alp's notes

I really like OpenVZ and Proxmox. But what I hate is programs which try to think for me. For example, we have /var/lib/vz and subdirectories on ZFS. If by chance it was not mounted on system startup, OpenVZ creates subdirectories in /var/lib/vz/template. Next time ZFS filesystem containers/vz/template just will not be mounted on non-empty /var/lib/vz/template. And you discover it only during runtime. If OpenVZ just threw error on startup, it would be better. If ZFS just silently mounted filesystems over non-empty directories (as usual mount does), it would be better. But two subsystems try to think for me and make my life better. I hate programs being so smart...

On attaching zones and linked images alp's notes

Recently updated IPS was added to OI and it caused some inconvenience to developers. The most annoying "feature" is caused by publisher check: NGZ first several publishers should be the same as GZ's publishers and their stickiness should match.
OK, I've set the publishers according to this rule, but today was surprised by fact that I can't longer set publisher to be non-sticky in NGZ. I've set publisher to non-sticky in GZ, NGZ, but still IPS complains on "pkg install":

pkg install: Invalid child image publisher configuration.  Child image publisher
configuration must be a superset of the parent image publisher configuration.
Please update the child publisher configuration to match the parent.  If the
child image is a zone this can be done automatically by detaching and
attaching the zone.

The parent image has the following enabled publishers:
    PUBLISHER 1: userland (non-sticky)

The child image has the following enabled publishers:
    PUBLISHER 0: (non-sticky)
    PUBLISHER 1: userland (non-sticky)
    PUBLISHER 2: hipster-encumbered

I've rechecked. Both GZ and NGZ had publisher set to non-sticky.  I've followed IPS advice - detached and attached zone. And zone attach -u failed with the same error.

OK. Time for black magic. I've unset userland publisher. Set GZ and NGZ publisher list only  to (non-sticky). The same issue - IPS doesn't see that GZ publisher is non-sticky now. So, I concluded that  information about parent image is recorded or cached in some local zone configuration file. Looked at zone's /var/pkg and found /var/pkg/linked/linked_ppubs, which listed [["", true], [userland, false]]. I changed this file to [["", false]] and after that could attach zone.

I think I still doesn't understand as linked images work (or should work), but they are starting annoying me...

Hipster 2015.03 is here alp's notes

We released our last snapshot ISO almost half a year ago. I believe, you want something new. You'll get it. New ISOs were just uploaded to dlc server. Let's see what has changed.

First of all, most evident changes were made in desktop area. We've updated Xorg server and libraries, which allowed us to incorporate some important security fixes from Oracle x-s12-clone and Debian Xorg. Also we've moved much more closely to Gnome 2.32. Most packages were updated to this level, excluding packages which either have a lot of specific patches (like gdm) or just dropped some significant functionality (like cheese, which dropped HAL support in version 2.32). Not everything has gone smoothly. We had to drop trusted desktop support during update. I believe nobody seriously used it under OI. The most annoying thing is that updated Xorg and Intel driver require some DRM updates, which are still not ready. So, if you have Intel video card, either pkg freeze X-incorporation and xorg, or use vesa driver.

General system changes

All Sun Studio-compiled C++ libraries were removed from the system. The libraries were published in their current form to, so you can grub necessary libraries and LD_PRELOAD them or use in alternative path if necessary. All X/g++/Y packages are renamed to X/Y and moved from /usr/g++ to /usr. We continue delivering system/library/c++/sunpro for the foreseeable future.
Text installer was changed to install OI on EFI-labeled disk by default. Note, in this case the entire disk is erased. If you want to install OI on MBR-labeled disk, choose partitioned install.

Desktop software and libraries

  • A lot of desktop libraries were updated
    • GTK2 is updated to 2.24.27
    • libdrm is updated to 2.4.58
    • libX11 is updated to 1.6.2, xcb support is enabled in libX11
    • xf86-video-ati driver updated to 6.4.16
    • nvidia proprietary driver was updated to 340.76
    • Mesa is updated to 10.5.1
    • Xorg is updated to 1.12.4. This requires updating xorg drivers and modules. OI-shipped modules will be updated automatically, but if you use VirtualBox, you'll have to update your guest additions to at least 4.3.22 version.
    • Glib2 is updated to 2.43.4
  • Enlightenment 0.19.3 is added as alternative desktop environment
  • fontconfig was updated to 2.11.1
  • libid3tag and libmtp were imported from SFE, gmtp is added
  • rdesktop is updated to 1.8.3
  • transmission is updated to 2.52
  • XScreensaver is updated to 5.32
  • gnome-commander is updated to 1.4.5
  • QT 4.8.6 is added
  • emacs is updated to 24.3
  • Input Method Selector was added from upstream input-method gate. Bug in svc:/application/desktop-cache/input-method-cache:default service preventing correct input methods functioning in recent OI was fixed. In fact, gtk input modules cache was moved from /etc/(amd64/)gtk-2.0/gtk.immodules to /usr/lib/(amd64/)gtk-2.0/2.10.0/immodules.cache and service has to regenerate these cache files in new locations . So, after update you can safely remove /etc/(amd64/)gtk-2.0/gtk.immodules.

Development tools

  • Subversion is updated to 1.7.19
  • SQLite is updated to
  • Python 3.4 is updated to 3.4.3
  • Binutils are updated to 2.25
  • OpenBLAS 0.2.13 is added
  • Mercurial is updated to 3.3
  • Ruby 1.9.3 is added
  • Ruby 1.8 is marked obsolete, all OI software is switched to Ruby 1.9.3
  • Ruby 2.2.1 is added
  • Curl is updated to 7.39
  • links are moved to /usr/lib(/amd64)
  • gawk is updated to 4.0.2, this fixes issues with pkgsrc bootstrap
  • MPICH is updated to 3.1.3
  • Sun Studio indent in /usr/bin was replaced by GNU indent. Old one is preserved in /opt/sunstudio12.1/prod/bin/indent

Server software

  • A lot of packages were updated, including apache 2.4, php 5.4, php 5.5, postgresql 9.3, samba 3.6, mariadb 5.5
  • PostgreSQL 9.4 is added
  • PostgreSQL 8.4 is marked obsolete
  • ISC dhcp server is updated to 4.2.7
  • BIND DNS server is updated to 9.9.6-P2
  • rsyslog is updated to 7.4.10
  • NTP is updated to 4.2.8p1

As always, we are proud to deliver to you latest illumos-gate bits.

There's also a lot of security fixes and small bug fixes.

Of course, I had more ideas than spare time, so some of them were not implemented. We still don't have PHP 5.6 and our OpenOffice package still doesn't work with XML-based document formats. I've looked at replacing cpp with one based on Schilix version, but unfortunately I found it to be not always compatible with Sun cpp. So, I've chosen preserve the status quo and we still deliver Sun cpp. I still would like to see postfix as first class MTA in OI.

I'd like to share some more ideas, which attract me now. First of all, we consider further updating of Xorg and other former xnv components (libXfont, freetype). GCC update is also on the roadmap. Our old samba and cups versions, dependency on python 2.6 and ageing Perl 5.16 make me sad.  Of course, I'd like to see PHP 5.6 in oi-userland and perhaps even look at hhvm.

Trip to Japan. Part one - Tokyo alp's notes

So, I've finally returned from Japan. I'd like to share my "road-diary". I need some time to arrange my records, so I'll split them in several parts.

Day 1

First day in the way. I'm going to Moscow as there's no direct flight to Tokyo. RZD surprised me giving ticket for 37 place in the car where there were only 36 places. In such way they sell places in conductor's compartment. Another pleasant surprise was that I had a ticket with included dinner. However, I haven't understood why they brought cake before chicken...

Day 2

One more day in the way. Moscow is as always crowded. After waiting for about 6 hours for my plane in Domodedovo and 9 hours night flight I'm feeling sleepy. Funny case in the plane. I've ordered tea. Got a bottle of gin. Asked "is it a tea?". Got reply, yes, this is tea. Hm… At home I'd like such tea, but in the road I'd like to avoid drinking alcohol. Luckily, another stewardess understood me better.

Day 3

The morning begun sharply and suddenly with Sun blinding me from the . window. The weather was perfect. I saw blue ocean and green land. I had to fill two blanks for foreigners and was afraid that I've done something wrong. However, custom officer just looked that all fields are filled and asked for the intended duration of my stay in Japan.
As for yens, we changed money in the airport. 1 dollars is roughly equivalent to 105 yen. To convert prices from yens you can divide by 100 and get price in dollars (or by 2.5 and get prices in roubles).
The first shock is that not everyone speaks English. Expected, but unpleasant. Luckily, even when people don't understand you, they always try to help. But more on this later. Now we were moving to the Tokyo television tower. Precisely, to the Buddhist temple by the tower. They say that the tower itself was constructed as a copy of Eiffel Tower. Don't know, it seems similar, but different at the same time.
There was a concert in front of the main temple, so we entered smaller one. Today is a Culture day and at the same time the birthday of Meiji Emperor (the one who like Peter the Great in Russia cut a window through to Europe for Japan). We've visited service in the temple. Buddhists are practical people - if you pay for the service, the monks will held one for you even if you are not a Buddhist. I've took a picture of the place where people tie ribbons with monks guidances. All predictions, guidances, amulets are valid only till the end of the year. You have to return them to the temple till the end of December and get new ones in January.
There are also statues in temple's garden. You can order one for you. This statue is devoted to the childrens of your family. You should dress them in caps, bring them toys and candles for your children to prosper. Periodically caps and toys are renewed. Statues stay in the temple's garden forever. And burial urns in the local cemetery - no. You should pay significant rent for the monument on it. Each monument holds several urns, perhaps, for the whole family. If no one pays a rent, the place is sold and urns are transferred to the common store inside the temple. On the photo you can see some ski-like sticks. They mark the number of prayers which were held for the dead.
There are also shogun graves in this cemetery. But this part of cemetery is opened for visitors only once per year. In the common part of the cemetery there is one outstanding tomb - the tomb of Emperor's wife who tried to prevent bloodshed between 14th shogun and Emperor in 19th century, when Emperor brought back true power from shoguns.
The last part of the day was devoted to visiting thermae. As I don't like public baths, I just read the book in the common place ahead of baths. Japanese in kimono look nice. But I think it wouldn't suit me :)
A real surprise was non-european sockets. And lavatory pan with a dozen of buttons (it was hard to find out the main one :). In the evening I've went to Indian Cafe. Interesting experience. I've ordered meat with spinach. When I got it there were more spices than meat. Seriously, I firstly thought that it was some sauce for the meat…
After buying a phone charger found interesting device in the hotel room - a charger for everything (this device has a lot of connectors for every gadget). Unluckily it didn't suit to my camera and I had to ask for Japanese-European adapter at hotel registry.

Day 4

In the morning we went to fish market. The main fish is tunny. The largest part of fish are sold in the morning, starting from 4 a.m. to the large customers. Later, starting from 8 a.m. small quantities of fish are sold to local restaurants. You can see a lot of customers in the market in the morning. I expected to see more fish, but it seems not all fish is shown on the counter. However, there are some exotic goods - crabs, calamars, seaweed.
Then we visited garden near Emperor's palace. However, we were not allowed to enter deep into the garden, because there were some officials in the palace. So, we went to Shinto shrine. We were lucky enough to see traditional marriage. Also, near the temple, we saw girls in traditional costumes. At some age girls should be blessed in the temple. This particular temple was devoted to Meiji Emperor. Near the temple there are two lines of barrels for Sake and wine. On the barrels there are labels of wineries which make donations to the temple.
Later we were visiting viewpoint at the 45th floor of some administrative building. They say the design of the building was inspired by Notre Dame de Paris. And they plainly have something common.
After dinner we were brought to one more temple (this time Buddhist). On the ceiling of the temple you can see the dragon fulfilling wishes of the congregation.
Then we visited tea ceremony. My groupmates were whining that it was too long. But I liked it - at last something national and colorful. In the evening I joined several groupmates and we were walking along Ginza. One of them was citing the poems she wrote. They seem rather nice.

Day 5

In Tokyo we were staying in Grand Prince Hotel Takanawa. Really nice hotel. It has a beautiful park. The park even has a pond with big and vivid fish. There is a small Buddhist chapel in the park.
Today we went to some lake by Mount Fuji. The view is picturesque, but today is rather cold here. Why we were sailing across the lake, I became frozen. The ship we were sailing at was made in some medieval style - with sails and cannons. There even were some pirates on the ship which begged money for making photos with them. During our journey I saw several shrines and temples on the banks of the lake. I tried to find some alcohol in nearby shops, because was frozen, but without any luck. Only when we went up by rope-way, I had some luck in finding sake.
We saw the valley of geysers. Local people boil eggs in their water. As water contains a lot of iron, the eggs become black (only the shell, the egg itself is usual). They say that such egg prolong your life by 7 years. Don't know, I haven't eaten it. The eggs are sold to tourists in any forms: the eggs themselves, the toys in form of eggs, the chocolate in form of eggs and so on.
The next stop on our way was at the house of geishas. A strange art. I can't understand what they study for 20 years. But resulting performance is beautiful. However, the music is a bit mournful (reminds me of Mongolian music).
The last point of our program for today was one more public baths. As I don't like it, I walked by the nearest village. The village itself resembles my mother's homeland. There are buildings on both side of the road. Small houses - hotels, museum. Mountains surround the village. Pine trees grow surround the route. The difference is that there are much more trees and expensive cars on the road.
On the way back the guide has told us about life in Japan. I don't like to live such life: marriage matchmaking, working 24 hours per day, wifes manage the house. There are no free high education. Scholarship should be returned after graduating. And, of course, Russian girls are prettier :)

There and back again alp's notes

Once upon a time in a galaxy far far away there has been OpenSolaris distribution. This was the first Unix-like desktop which allowed me to completely replace Windows on my desktop. It wasn't perfect, but a magic of ZFS and IPS made it very attractive. As we now, the sun sets... So, with the end of OpenSolaris era I've moved to FreeBSD and has never regretted about it.

But I always liked illumos and was interested in OpenIndiana distribution. So, now, when I seem to be one of the last interested in it, I tried to go back. Really, migration was smooth. I've bought Nvidia GeForce 740-based video adapter, 4 more GB of RAM and after this I was sure that my hardware is supported by OI.

OI could recognize FreeBSD ZFS pool, so I've detached one disk from zpool, installed OI Hipster from October ISO, imported data and looked at software. As always, software choice for OI if you don't want to compile it yourself is not wast. Yes, there are pkgsrc builds, but they seems foreign in IPS world. So, I used sfe and sfe-encumbered repositories for OI /dev. I hope to use vlc, but vlc 1.1 from sfe-encumbered was a complete garbage. It didn't want to play anything or when it played something it was awful (like no sound or no video or no navigation in DVD menu...) So I stopped on totem/rhythmbox combination and used additional gstreamer codecs from sfe-encumbered. This works, however you should set sfe-encumbered before sfe in your publisher's list.

When speaking about OI one should mention IIMF/IIMD and other IIIM shit. Luckily, it can be turned off. I've finished just adding setxkbmap command to the list of startup applications.

Unfortunately, our Apache OpenOffice from /hipster is still buggy, so I had to use one from AdfinisSyGroup. VirtualBox 4.3.18 from Oracle for Solaris/x64 works fine. I was pleased with our Firefox 24.8.1. Adobe flash plugin 10.1 r85 sometimes crashes. Especially annoying is that it crashes on the flash games which I have to support. At least I've expected that it would not work - the application require Flash 11+. But for other use cases it works. Having working evince, xchm and FBreader seems enough to read documentation. zpool resilvering has completed by 50%. So, now I'm eating my own dog's food.

Hipster 2014.10 is finally out alp's notes

We are finally ready to present you new OpenIndiana Hipster snapshot and ISO images. We had some difficulties with generating installable "entire", so we had to delay snapshot for a week. But now you can just pkg fix entire to be more stable in constantly-changing Hipster world :) Since last snapshot a lot of work was done. I tried to summarize the changes in this informal "release notes". Please, be sure to read at least Perl-related notices before updating from previous snapshot.

General system changes

We performed migration from GCC 4.7.4 to GCC 4.8.3. New packages are compiled with GCC 4.8.4.
Perl 5.10 is not compulsory now. All perl dependencies were updated to use Perl 5.16 or don't care about perl version. This includes changes to illumos-gate. /usr/perl5/bin/perl now is mediated symlink, pointing to perl 5.16 by default. However, if you would like to compile unmodified illumos gate, you should switch it back to perl 5.10:
pkg set-mediator system-perl=5.10
Perl 5.10 modules which are required to compile illumos-gate are preserved. Other perl 5.10 modules are removed. If you have perl-510 installed on your system and don't need it, you can just remove runtime/perl-510, runtime/perl-510/extra and runtime/perl-510/module/sun-solaris.
During perl update version of perl-516/module/sun-solaris was DECREASED from 5.11 to 0.5.11 for consistency with other illumos-gate provided software. So if you have installed it, please, remove it BEFORE updating to new Hipster snapshot.
Perl 5.16 was recompiled without -Dperl_static_inline="static" flag to avoid creating one more patch for illumos-gate (this can affect perl ABI). So, if you had self-compiled perl modules, possibly, you have to recompile them.
Also, for consistency we renamed library/perl-5/xml-parser@5.12.1- to library/perl-5/xml-parser@2.41. So, if you have it installed (and every desktop system has it), please update it to library/perl-5/xml-parser@2.41,5.11-2014.1.2.0 (pkg update library/perl-5/xml-parser@2.41,5.11-2014.1.2.0). Also you'll have to update desktop/system-monitor/gnome-system-monitor if you have it installed.

Development tools and compilers

  • OpenJDK was updated to 1.7.60. GCC 4.7 was updated to 4.7.4. GCC 4.8.3 and Clang 3.4 were added.
  • GDB was updated to 7.6.2
  • Mercurial was updated to 3.0.2
  • MPICH was updated to 3.1.2
  • ant was updated to 1.9.3
  • python 2.7 was updated to 2.7.8

Common software

  • bash was updated to fix latest bash CVEs, GNU coreutils were updated to 8.22, CUPS updated to 1.4.8, doxygen updated to 1.8.7, GNU grep was updated to 2.20, gnupg to 2.0.25
  • Several packages to work with numerical data were added (datamash, hdf5)

Server software

  • A lot of packages were updated, including apache 2.4, apache 2.2, nginx, php 5.4, php 5.5, squid 3.1, tomcat 6.0, postgresql 8.4/9.3
  • Ldap backend was enabled for OpenLDAP server.
  • Illumos-gate provided wu-ftpd was replaced with proftpd 1.3.5
  • Barman was updated to 1.3.3
  • NTP was updated to 4.2.7p453
  • Bind was updated to 9.9.6

Desktop software

  • Firefox and Thunderbird were updated to 24.8.1
  • Packages from sic-team incorporation (notably, Mozilla nss and nspr) were updated.
  • glib-networking, webkit were added
  • Experimental package for Apache OpenOffice 4.1.1 was added. There is known issue with it - it can't create ODF documents. We are working on it.
  • ntfsprogs were updated to 2014.2.15 version
  • DJVU support was added to evince
  • Gnome-pilot and Gnome-pilot-link packages are obsolete now as upstream is dead.

Also there were lot of other small fixes.

As always we ship the newest illumos-gate version, so you can leverage the great work of illumos developers.

Last time I blogged about Hipster I shared some plans. From things which were planned but not made I have to point to integration of new Perl version and 64-bit Perl version. Also we didn't add vlc and other multimedia software to oi-userland because of legal issues. We still miss fsvs. However, I hope we'll find decision for this problem soon.

Now I'd like to share my current plans, the things I'm interested in and the things I'm working on.

First of all, I'd like to look at OpenOffice issue with ODF files. It seriously bugs me, but I don't know if and when I'll be able to solve it.

One thing which annoys me in OpenIndiana desktop is lack of text search in gnome terminal. I have prepared update to gnome-terminal 2.32. It's coming soon.

Also I'd like to look at migration of some XNV components to oi-userland. This should allow us to enhance their packaging (so that it better correlates with oi-userland build system) and later I hope to update it.

I'm seriously annoyed by our out-of-date sendmail, coming from illumos-gate. I'd like to see postfix as a first-class OI MTA. Of course, we'll also need dovecot and perhaps some other mail server software.

Finding security patches for our squid 3.1 is becoming harder. Perhaps we have to update it to recent 3.4 version and also receive SMP support as a pleasant triffle.

PostgreSQL 9.4 release is coming. I want to have it in the gate. On other hand, I think we should remove PostgreSQL 8.4 once 9.4 is landed. PostgreSQL 8.4 has already reached its EOL.

PHP 5.6 is already out. I think, we'll get it soon. As always I hope to use EveryCity's work :)

Our Ruby is amazingly old. We have to migrate to at least ruby 1.9. Ruby 1.9 package is ready and just waiting for official snapshot announcement

I'd like to see some binary blobs disappearing from OI Hipster - like cpp (we can use Joyent version), also I hope once we'll have open source gcc-compiled dmake alternative.

Adding dpkg/apt tools and support for DilOS zones in OI seems attractive and perhaps even necessary thing if we want to better collaborate with DilOS on userland packages.

I hate Roskomnadzor alp's notes

I don't have good words for them. This organization of bureaucrats and idiots are a real pain in the ass. Yesterday we (South Federal University) had to block all Youtube IP addresses, because our chiefs are afraid of possible fines. We have Internet Provider license and so have to filter "unacceptable" content. After last Roskomnadzor representatives found that several links on youtube, providing extrimist materials, are accessible from our network. We don't have any filtering services, so we were told just to ban all Youtube IP addresses to avoid penalties. What a hell! I've already received a lot of pleasant comments from our users. Really, ignorance is strength!

rude hack to proceed on zoneadm attach error alp's notes

I have two zones on my build host. One is build zone, serving IPS repository for the whole host,  so I have to be very careful with its updates and another - test one. I wished to update test zone, so issued

# zoneadm -z zonename detach
# zoneadm -z zonename attach -u
and noticed that I detached build zone with repository. zoneadm launched pkg, pkg worked for a while, and then it said:

Evaluation: Packages in zone zonename are out of sync with the global zone. To proceed, retry with the -u flag.
Result: Attach Failed.

What a hell! NGZ and GZ were in sync... At least both of them were latest /hipster. So I removed all publishers served by this zone from host and zone config.  The same reaction.
After grepping for this message in  /usr/lib/brand/ipkg/attach I found that this message is produced in  the following part of the script ($m_need_update message).
# Bring the ngz entire incorporation into sync with the gz as follows:
# - First compare the existence of entire in both global and non-global
# zone and update the non-global zone accordingly.
# - Then, if updates aren't allowed check if we can attach because no
# updates are required. If we can, then we are finished.
# - Finally, we know we can do updates and they are required, so update
# all the non-global zone incorporations using the list we gathered
# from the global zone earlier.

if [[ -z $gz_entire_fmri && -n $ngz_entire_fmri ]]; then
if [[ $allow_update == 1 ]]; then
LC_ALL=C $PKG uninstall entire || pkg_err_check "$f_update"
log "\n$m_need_update" "$ZONENAME"

if [[ $allow_update == 0 ]]; then
LC_ALL=C $PKG install --accept --no-refresh -n $incorp_list
if [[ $? == 0 ]]; then
log "\n$m_complete"
log "\n$m_need_update" "$ZONENAME"

I've just commented all these checks out and after this zone attach succeed. Zone is working now and I'm glad I don't have to reinstall my build zone....

Experience: dialog with prosecutor alp's notes

Wow! Today we had a guest from prosecutor's office.
They checked if we (South Federal University) ban sites from prosecutor's list. Luckily, our upstream provider does it for us.
But a check was ridiculous.
Yes, we receive daily lists of sites to ban. I though they would check some sites from this list and go in peace. But prosecutor just searched for prohibited works with Google and tried if she can download the materials. Of course, Google found working links :)
What a hell? Why should we imitate some work if everyone knows how to avoid these regulation rules? Why should anyone spend resources for content filtering? I think our government are a herd of archaic dinosaurs who just don't know how to lick chief's arse better

OpenIndiana /hipster progress and my long TODO list... alp's notes

There is always a lot of things to do. Of course I'd like to see OpenIndiana a modern universal OS, but it's still a long way to go. We've made a lot of work in the last two monthes.
1) Thanks to Adam Stevko and  Andrzej Szeszo we have a modern IPS version. Unfortunately, IPS GUI has gone, but as it was dropped even by upstream , it's not a big loss. We got ability to generate dependencies on mediated links, improved speed of operations and I hope fixed a number of bugs.
2) Andrzej Szeszo has updated NVidia drivers to version 331.20
3) Andrzej Szeszo has finally proposed a reasonable package versioning scheme, which allows to do /hipster more stable and predictable. I hope we'll adopt it soon.
4) I continued my work on JDS conversion and updates: we received Python 2.7, a lot of Python modules and several GUI applications (totem, rhythmbox, fbreader) were moved from JDS or added to oi-userland. The most noticable additions are OpenJDK 1.7.45, Firefox 17esr and Bacula 5.2.13.
I know, there should be some bugs there, but I hope we'll deal with them soon.

Now I'm interested in several issues:
1) I'm currently working on Thunderbird update to 24.2.0. I hope to finish in about 10 days if there are no any surprises.
2) The second thing I'd like to do is to work a bit on Python 2.7 modules so we'll be able to turn on compilation of Python 2.7 components (I mean setuptools, cherrypy,  mysql, psycopg and so on) by default. As I'm on Python I would deprecate Python 2.4, 2.5. It also worth investigating if IPS could use Python 2.7.
3) I'd like to look on PHP 5.5. Jon Tibble has recently added PHP 5.5 to ec-userland. We could base oi-userland component on this version.
4) I think I should finally build, test and integrate brasero to userland (Ken Mays has prepared patches and Makefile long ago)
4) I would really like to move to userland some more things from oi-build, first of all, kvm and qemu, but I don't think I have a hardware necessary to test it.
5) Also I'd like to look at perl - what do we need to allow Perl 5.10 go away? We definately need 64-bit Perl version. It worth to look at Perl update. One interesting part is current work of Andrew Stormont on which can allow us to use Perl 5.16 for building illumos-gate.
6) I also want to see apache 2.4 in the gate ( ). However, I currently don't have a clear view if it can coexist with apache 2.2. The main issue I see now is php module - php 5.4 component currently doesn't allow to build apache php module for several apache versions.
7) And every operating system would like to be a decent desktop. I'd like to see fusefs and vlc out of the box....

It's a long list, I don't know how long it will take to implement everything, but I'm going to do at least something :)

beadm destroy error alp's notes

Today when I was trying to destroy old boot environment, I got strange error:

# beadm destroy oi-hipster-2013-08-06
Are you sure you want to destroy oi-hipster-2013-08-06?
This action cannot be undone (y/[n]): y
be_destroy_callback: failed to destroy data/zones/build/ROOT/zbe: dataset is busy
be_destroy: failed to destroy BE data/zones/build/ROOT/zbe
be_destroy_zone_root_callback: failed to destroy zone root data/zones/build/ROOT/zbe
be_destroy_zone_roots: failed to destroy zone roots under zonepath dataset data/zones/build: dataset is busy
be_destroy_zones: failed to find and destroy zone roots for zone build
be_destroy: failed to destroy one or more zones for BE oi-hipster-2013-08-06
I didn't want to destroy zone root FS accidentally, so was a bit scared. However, after looking at it a bit longer, I found out, that zone root FS has several manual ZFS snapshots. After destroying snapshots I was able to destroy BE.

Multithreaded Rust on Threadripper Lice!

I recently ran some benchmarks on a Threadripper 3960X system and the results were surprising me quite a bit. Simplified, the throughput the benchmark recorded went down, from 341 MB/s on a MBP to 136 MB/s on a Threadripper desktop. Prior I had read Daniel Lemire’s notes on the sub optimal performance for simdjson on Zen 2, which is heavily used in the benchmark, but the suggested drop were a few percent not half.

Long story short, this made me curious what caused this. First stop: perf.

perf crossbeam

Notice the first item? It is crossbeam_channel::flavors::array::Channel<T>::recv. Oh my, I never saw that one hogging so much cpu time, in fact we spend more time in receiving from the channel then we spend in parsing or serializing JSON!

Lets add a bit of Threadripper trivia, the design AMD went with was splitting the CPU from a single silicon to multiple small dies, they call CCDs with in turn are consists of two CCXs that then contain the cores and level 1-3 cache. So lets look at another thing, htop (trusty little tool to show our load):

htop load on cores

In this screenshot we can spot that one thread seems to be running on the 5th core, one on the 16th and one on the 19th and 20th. Thinking back to the design of the Threadripper this is a bit of a hint, those cores are on different CCXs and even further on different CCDs so what happens if they were on the same?

Boom 400+ MB/s! taskset -c 0,1,2 does the trick, that’s a really nice improvement and looking at the perf output we can see recv to move from nearly 11% of CPU time to 7.28%, now that’s a neat improvement. Not only is it nearly 3x faster then the first benchmark but also is it 20% faster then on the laptop. So far so good.

perf crossbeam

But it’s still leaves the question, why and if we can do something about this. Enter a little benchmark and look at what it puts out for the first core (it’s a lot of output otherwise).

B 0 -  0: -
B 0 -  1: 818us/send
B 0 -  2: 673us/send
B 0 -  3: 2839us/send
B 0 -  4: 2421us/send
B 0 -  5: 2816us/send
B 0 -  6: 3466us/send
B 0 -  7: 3634us/send
B 0 -  8: 3267us/send
B 0 -  9: 3042us/send
B 0 - 10: 3633us/send
B 0 - 11: 3535us/send
B 0 - 12: 3334us/send
B 0 - 13: 3443us/send
B 0 - 14: 3348us/send
B 0 - 15: 3398us/send
B 0 - 16: 3459us/send
B 0 - 17: 3108us/send
B 0 - 18: 3287us/send
B 0 - 19: 3393us/send
B 0 - 20: 3369us/send
B 0 - 21: 3248us/send
B 0 - 22: 3290us/send
B 0 - 23: 3323us/send

B 0 - 24: 487us/send
B 0 - 25: 812us/send
B 0 - 26: 676us/send
B 0 - 27: 2859us/send
B 0 - 28: 2853us/send
B 0 - 29: 2864us/send
B 0 - 30: 3475us/send
B 0 - 31: 3620us/send
B 0 - 32: 3582us/send
B 0 - 33: 3497us/send
B 0 - 34: 3524us/send
B 0 - 35: 3488us/send
B 0 - 36: 3331us/send
B 0 - 37: 3303us/send
B 0 - 38: 3365us/send
B 0 - 39: 3333us/send
B 0 - 40: 3324us/send
B 0 - 41: 3363us/send
B 0 - 42: 3554us/send
B 0 - 43: 3351us/send
B 0 - 44: 3207us/send
B 0 - 45: 3240us/send
B 0 - 46: 3377us/send
B 0 - 47: 3275us/send

First things first, the numbers here are 0 indexed, unlike in htop where they’re 1 indexed. So core 0 here means core 1 in htop. The test runs only for a second per core combination (as it goes through all cores and otherwise takes a really long time), some variation is to be expected. That gets really slow really fast. We can see that core 24-47 are the SMTs cores on the physical cores 0-23, so 24 being the second thread on core 0. The second observation is that core 0-2 are in the same CCX, performance is reasonable fast here. 3-5 seem to be on the same CCD and so on.

Lets look at the code for the crossbeam channel. The part that’s interesting is that both head and tail are wrapped in CachePadded. Fortunately I have a friend who keeps going on about false sharing whenever performance becomes a topic so that was a really good hint here. Looking through the struct aligning head and tail to the cache line makes a lot of sense they’re frequently accessed from both sides of the queue but there is another part that’s frequently used on both sides. The buffer, and that is just an array of T so it might not align well to the cache. In other words, if we access buffer[x] we might invalidate buffer[x-1] or buffer[x+1] (or more). So what happens if we wrap the the elements in a CachePadded?. The result looks quite nice, it cut down by 50% when going over CCX boundaries:

B 0 -  0: -
B 0 -  1: 630us/send
B 0 -  2: 678us/send
B 0 -  3: 1319us/send
B 0 -  4: 1256us/send
B 0 -  5: 1291us/send
B 0 -  6: 1438us/send
B 0 -  7: 1504us/send
B 0 -  8: 1525us/send
B 0 -  9: 1660us/send
B 0 - 10: 1772us/send
B 0 - 11: 1807us/send
B 0 - 12: 1382us/send
B 0 - 13: 1380us/send
B 0 - 14: 1387us/send
B 0 - 15: 1375us/send
B 0 - 16: 1382us/send
B 0 - 17: 1383us/send
B 0 - 18: 1471us/send
B 0 - 19: 1471us/send
B 0 - 20: 1463us/send
B 0 - 21: 1462us/send
B 0 - 22: 1468us/send
B 0 - 23: 1457us/send

B 0 - 24: 466us/send
B 0 - 25: 619us/send
B 0 - 26: 671us/send
B 0 - 27: 1438us/send
B 0 - 28: 1422us/send
B 0 - 29: 1514us/send
B 0 - 30: 1789us/send
B 0 - 31: 1688us/send
B 0 - 32: 1812us/send
B 0 - 33: 1820us/send
B 0 - 34: 1719us/send
B 0 - 35: 1797us/send
B 0 - 36: 1383us/send
B 0 - 37: 1364us/send
B 0 - 38: 1373us/send
B 0 - 39: 1383us/send
B 0 - 40: 1370us/send
B 0 - 41: 1390us/send
B 0 - 42: 1468us/send
B 0 - 43: 1467us/send
B 0 - 44: 1464us/send
B 0 - 45: 1463us/send
B 0 - 46: 1475us/send
B 0 - 47: 1467us/send

With all of this, the code went from 136 MB/s to over 150 MB/s when not pinned to cores, while this isn’t close to where I’d like to to be, it is a 10% improvement in throughput. And looking at perf again recv is completely gone from the list, which is nice!

perf crossbeam

This is the conclusion for now, if I have more interesting finds I’ll add a continuation - so I’ll keep digging.