Tribblix - minimal plus pkgsrc The Trouble with Tribbles...

The latest update (0m20.5) of Tribblix is now available for download.

This one comes relatively quickly after its predecessor, but includes the eager fpu and ldt fixes.

The visible change this time is that there's a minimal ISO available. At just over 200M, it's about a fifth of the size of the regular ISO.

The only difference between the regular and minimal ISO is in what extra packages are dropped on the ISO. If you aren't going to add overlays at install, then the minimal ISO is much better.

The use cases for the minimal ISO are the obvious minimal server installs (including cloud-based), but also if you decide to use pkgsrc for your software rather than Tribblix native packages.

Tribblix is all about choice and flexibility, so if you want to use pkgsrc, go right ahead. You'll get a broader choice of software, certainly. In some areas you'll get newer versions than I provide, in some areas Tribblix will update more aggressively.

One thing I have noticed, though, is that if you want to use pkgsrc, do so exclusively. Mixing and matching native packages and pkgsrc really doesn't work.

With that in mind, the minimal ISO makes a great base for pkgsrc. To make it even easier, the pkgsrc bootstrap is available on the ISO itself. So if you want to install Tribblix with pkgsrc, just invoke the installer like so:

./live_install.sh -B c1t0d0 pkgsrc

Then, as root

export PATH=/opt/local/bin:$PATH

and you're all set to install packages from pkgsrc.

To get the X server and basics:

pkgin install modular-xorg

To install a graphical desktop, such as Xfce - there are many others, of course:

pkgin install xfce4

And then, as a regular user

export PATH=/opt/local/bin:$PATH
startxfce4

And, lo and behold, you should have a functional basic Xfce desktop running.

Prague & IETF Josef "Jeff" Sipek

This post is part of a series named “Europe 2017” where I share photos from my adventures in Europe during the summer 2017.

In mid-July 2017, I got to travel to Prague to participate in IETF’s 99th meeting.

The IETF meeting itself was great—lots of useful discussion about the next-generation email accessing protocol (called JMAP).

I stayed a couple of days extra to enjoy Prague, and Holly flew out from Helsinki to revisit Prague where she’s been once before—for our honeymoon.

I dragged my D750 and the two lenses with me and made to sure to take photos (almost) all the time. The gallery contains only a handful of the approximately 1100 raw files. Of those, I selected 11 for this blahg post.

Wikipedia article: Malá Strana Bridge Tower:

Wikipedia article: St. Nicholas Church with the Wikipedia article: Žižkov Television Tower in the background:

Wikipedia article: Matthias Gate with Wikipedia article: St. Vitus Cathedral peeking in the background:

The Wikipedia article: National Theatre:

Wikipedia article: Charles Bridge and a view of Wikipedia article: Old Town:

Wikipedia article: St. Vitus Cathedral from Wikipedia article: Petřín near sunset:

St. Nicholas Church again during sunset (and without the ugly Žižkov TV tower):

A gargoyle keeping the St. Nicholas Church’s masonry safe:

A view from the top of Wikipedia article: Old Town Bridge Tower with roofs and towers of (left to right):

Church of Saint Wikipedia article: Francis of Assisi, the left tower of Wikipedia article: Church of Our Lady before Týn, the clock tower and the Astronomical tower of Wikipedia article: Clementinum:

St. Nicholas Church yet again, this time from the Malá Strana Bridge Tower:

Wikipedia article: Charles Bridge, Wikipedia article: Old Town Bridge Tower, Church of Saint Wikipedia article: Francis of Assisi, and Wikipedia article: Žižkov Television Tower (from the Malá Strana Bridge Tower):

Prague offers a lot to see. The few photos I selected for this blahg post don’t show anywhere near enough of it. There are more photos in the gallery, but even those are merely highlights of what one can see in Prague. If you haven’t been to Prague yet, I highly recommend a trip.

Evaluating the Evaluation: Benchmarking Checklist Brendan Gregg's Blog

A co-worker introduced me to Craig Hanson and Pat Crain's performance mantras, which neatly summarize much of what we do in performance analysis and tuning. They are: **Performance mantras**

  1. Don't do it
  2. Do it, but don't do it again
  3. Do it less
  4. Do it later
  5. Do it when they're not looking
  6. Do it concurrently
  7. Do it cheaper
These have inspired me to summarize another performance activity: evaluating benchmark accuracy. Good benchmarking rewards investment in engineering that actually improves performance, but, unfortunately, poor benchmarking is a lot more common. I have spent a lot of my career refuting bad benchmarks, and have developed such a knack for it that prior employers adopted a rule that no benchmark can be published unless approved by me. Because benchmarking is so important for our industry, I'd like to share with you how I do it. Perhaps you're choosing products based on benchmark results, or building a product and publishing your own benchmarks. I recommend using the following benchmarking questions as a checklist: **Benchmarking checklist**
  1. Why not double?
  2. Did it break limits?
  3. Did it error?
  4. Does it reproduce?
  5. Does it matter?
  6. Did it even happen?
In more detail: ### 1. Why not double? If the benchmark reported 20k ops/sec, you should ask: why not 40k ops/sec? This is really asking "what's the limiter?" but in a way that motivates people to answer it. "What's the limiter?" sounds like a homework exercise of purely academic value. Finding the answer to "Why not double?" might even help you remove a limiter and double your benchmark numbers! ### 2. Did it break limits? This is a sanity test. I've refuted many benchmarks by showing that they would require a network throughput that would far exceed maximum network bandwidth (off by, for example, as much as 10x!). Networks, PCIe busses, CPU interconnects, memory busses, storage devices (both throughput and IOPS) – all have fixed limits. Networking is the easiest to check. In some cases, a benchmark may appear to exceed network bandwidth because it returns from a local memory cache instead of the remote target. If you test data transfers of a known size, and show their rate, you can do a simple sum to see the minimum throughput required, then compare that to network links. ### 3. Did it error? What percentage of operations errored? 0%, 1%, 100%? Benchmarks may be misconfigured, or they may be stress testing and pushing targets to error-inducing limits. Errors perform differently from normal operations, so a high error rate will skew the benchmark. If you develop a habit of reading only the operation rate and latency numbers from a lengthy benchmark report (or you have a shell script to do this that feeds a GUI), it's easy to miss other details in the report such as the error rate. ### 4. Does it reproduce? If you run the benchmark ten times, how consistent are the results? There may be variance (eg, due to caching or turbo boost) or perturbations (eg, system cron tasks, GC) that skew a single benchmark result. One way to check for system perturbations is via the load averages: let them reach "0.00, 0.00, 0.00" before you begin benchmarking, and if they don't get to zero, debug and find out why. ### 5. Does it matter? If I showed that the wheel on one car would fly apart if spun at 7,000 mph, and the wheel on another car at 9,000 mph, how much would that affect your purchasing decision? Probable very little! But I see this all the time in tech: benchmarks that spin some software component at unrealistic rates, in order to claim a "win" over a competitor. Automated kitchen-sink benchmarks can also be full of these. These kinds of benchmarks are misleading. You need to ask yourself: how much does this benchmark matter in the real world? ### 6. Did it even happen? Once, during a proof of concept, a client reported that latency was unacceptably high for the benchmark: over one second for each request! My colleague began analysis using a packet sniffer on the target, to see if packets were really taking that long to be returned by the server. He saw nothing. No packets. A firewall was blocking everything – the reported latency was the time it took the client to time out! Beware of misconfigurations where the benchmark doesn't actually run, but the benchmark client thinks it did. I hope you find this methodology useful, and feel free to forward this to anyone doing or evaluating benchmarks. I'll add this methodology to my online collection: [Performance Analysis Methodology]. [Performance Analysis Methodology]: www.brendangregg.com/methodology.html

Letter to Duncan Hunter (Immigration) /dev/dump

(Congressman Duncan Hunter is my Representative in the House.  Today I sent the letter below to him through the congressional email posting service (which verifies that I'm his constituent).  A copy is here for others to read.  I encourage everyone, especially those in districts with Republican congressional representation to write a similar letter and send it to their congressman via the site at house.gov -- you can look up your own representative on the same site.)

Congressman Hunter,

I have read your "position" statement with respect to the administrations "Zero Tolerance" treatment towards immigration, and the separation of families seeking asylum, and I am *most* dismayed by the position you have taken.

I would encourage you to start by reading the full text of the following court order, which describes the reprehensible actions taken by the administration: https://t.co/adhGO6BDJR

This is not hearsay, but legal findings by a US Court.

Your claim that "most asylum seekers" are not being broken up is disturbing, because it also indicates that you believe it is okay to separate families "sometimes".  The reality is that there are cases where separation is warranted to protect the children, but there is ample evidence that this is not the case in many of these separations.  (See the above court case.)

Not only that, but we know of cases where families have been separated for doing nothing wrong than presenting themselves fully legally at a border crossing and petitioning for asylum.

Even in misdemeanor cases of illegal entry, the punishment of breaking families up -- while both children and parents are held separately for deportation proceedings, is both cruel and unusual punishment, and entirely unnecessary.

It is also my belief that these separations create more risk to the United States, making us less safe.  Some of these children will remember the horrible ways that they were treated as criminals by our country.  How many will radicalize as a result later in life?  Just one is too many, and utterly unnecessary.

Furthermore, the use of the misery of these families as some kind of political card to achieve ends is both morally reprehensible and of dubious value -- the border wall is a boondoggle and will have little effective value. The real problem of immigration is the attraction of millions of jobs -- jobs that should be filled *legally*.  (So mandatory eVerify, with criminal prosecution against *EMPLOYERS* who violate rather than against poor immigrants is the real fix to that problem.  You backed mandatory eVerify, an action which I applaud.)

The ends -- a border wall -- do NOT justify the means, which have been amply and justifiably compared with the approach used by Nazi Germany in the 1930s, when dealing with Jewish families.

As a nation we are embarrassed by our Internment of Japanese Americans during the same time frame, and the US government has been called to account for that in the past.  Yet even in the midst of world war our leaders did not stoop to separating parents from the children, or to use children as some kind of pawns in a larger political scheme.

Indeed, these actions are more akin to those used by terrorists -- literally using "terror" (in this case fear of breaking a family up, which any parent knows is amongst the most terrible things to be contemplated) to achieve political ends.

Please think long and hard about your decision to stand with Trump in this regard.  If you stand with him here -- as an agent of terror and tyranny, then I cannot in good conscience stand with you.

You have a unique opportunity to break from the party lines, and demonstrate moral courage here -- an opportunity which if taken would certainly win back my support.   Or you can side with forces evil in supporting actions that numerous industry and business leaders have called "morally reprehensible" and "inhumane".

The choice is yours to make.  For now.


Self Publishing Lessons /dev/dump

Over the past several weeks I've learned far more than I ever wanted to about the self publishing process.  I'm posting some of my findings here in the hopes that they may help others. 

TLDR;


If you're going with eBooks, and you should, consider using an author website to sell it "early", and once your book is finished publish it with Kindle Direct Publishing and Smashwords.  Keep the author website / store up even after, so you can maximize returns.  Price your eBook between $2.99 and $9.99.

If you're going to go with Print, start with Amazon Kindle Direct Publishing first, unless you're only needing a small run of books printed only in the USA (in which case TheBookPatch.com looks good).  Once you're book is really done, and you're ready to branch out to see it available internationally and from other bookstores, publish it with Ingram Spark.

Get and use your own ISBNs (From MyIdentifiers.com -- buy 10 at a time), and make sure you opt out of Kindle Select!

More details are below.

eBook Formats


Let's start with ebook formats first.  Be aware that I'm writing this from California, with no "nexus" elsewhere, and electronic goods (when they are purely electronic) are not taxable here.  That means I haven't worried much about accounting for VAT or Sales Tax, because I don't have to.

Leanpub


Leanpub is how I started out, but they've altered there terms several times.  Right now their royalties are not substantially less than anyone else (they used to be), and you can get an even higher return selling through your own store (such as Selz.com).  The one thing they have over everyone else is their Markua tools, and their focus on helping authors in the early stages -- you can "publish" before the book is complete on Leanpub.  I'm not sure how useful Markua is to other people -- I don't use it at all.  If you have a book in progress, this will let you sell copies early.  But, they do take a cut -- 80%. Frankly, their business model seems a bit iffy right now, and I wouldn't like to put too many eggs in that basket.  You won't need an ISBN at Leanpub.  They pay 80% royalties, allow free updates (to a limit most authors are unlikely to hit).  Leanpub has very limited reach, and doesn't distribute to anywhere else.

Author Website


The cheapest and most cost effective way to sell your ebooks is to open your own author store.  Sites like Selz.com will let you do this for free, and only charge reasonable transaction fees.  With this approach you can get about 95% of the book sales.  You can publish as soon as you want, send updates as often as you want, and don't need ISBNs or anything like that. On the downside, you have to do a little more work to set things up.  You'll also have limited reach, and pretty much look like a fly by night operation.  If you want to "pre-publish" before a work is complete, this is a way to do that, without paying that 20% to Leanpub.  You can also leave this store open, and point to it from your personal author pages, even after you are working with the larger distributers.

Ingram Spark


Ingram Spark's ebook distribution service gets broad reach through relationships with the various outlets.  You can use them to get to Apple, Kobo, even Amazon.  And yet I would not recommend doing this.  First off they charge to set up the book, typically $25.  Then if you want to make a revision to the book, it's another $25.  And then you're typically going to set it up so that you get only 45% of the royalties (or 40% if you really messed up and didn't opt out of the Amazon agreement.) . Furthermore, I found that their conversion of my ePub to Kindle format was inferior, leading to a poor reading experience on those devices.  (I have some complex layout, and custom fonts, as the book is technical in nature.)   I had much better luck generating my own .mobi format and working with Amazon directly.   Their service takes forever to get back to you -- I'm still waiting while I try to remove my eBook from their distribution.  In short, I would not use Ingram Spark for eBook.  You also will need an ISBN if you use Ingram Spark.

Amazon (Kindle Direct Publishing)


Using the Kindle Direct Publishing was pretty easy, and this let me provide a .mobi file that was optimized for Kindle, addressing issues caused by converting from ePub.  (To be fair most authors won't have these problems.)  If you want to reach Kindle readers (and you do!), you should just set up a KDP account.  One word though -- don't opt-in to Kindle Select!!  Amazon is great, for distributing to Amazon customers.  But you don't want to give away your exclusivity.    There is a weird set of rules about royalties with KDP though.  If you want to get their best 70% (which won't be in all markets, but the main ones) you need to set your List Price between $2.99 and $9.99, inclusive.  (Other values are used for other currencies.)  Deducted from your 70% rate is the cost to transfer the data to the user, which turns out to be pretty cheap -- less than a dollar typically.  (But if you're only selling a $2.99 book, make sure you keep the file sizes down, or this will hurt your rates.)  You can opt for a flat 35% royalty instead, which might make sense if your book is heavy on content, and its required if your book is outside the price points.  (This is why you never see ebooks listed for $11.99 or something like that on Amazon.)

Smashwords


I just set up my account with Smashwords, and I'm thrilled so far.  It looks like you'll get about 80% royalties through their own store, and 60% if your book is bought through one of their partners -- which includes just about everyone -- including Apple, GooglePlay, Kobo, etc.  This gets you pretty much everywhere, except Amazon.  But you did set up a KDP account already right?  They take the royalty, and you're done.  There is one fairly severe draw back to Smashwords -- they want you to upload your manuscript as a specially formatted Word document.  (They do have direct ePub though, which you can use if you want.  I did this because I don't have a Word version of my book, and it would be difficult to get one -- it was authored in Asciidoctor.)   You will need an ISBN to get into their expanded distribution program, of course.  They will offer to sell you one, but I recommend you not do that and use your own.  (Especially if you're uploading your own ePub.)

Direct Retailer Accounts


You can maximize royalties by setting up direct accounts with companies like Apple, Kobo,  and Barnes&Noble.  In my experience, it just isn't worth it.  Dealing with these all is a headache, and it takes forever.  Some, like Google Play Store, are almost impossible to get into.  As the list gets large, the percentage of your distribution that are covered here diminishes, consider whether that extra 10% royalty rate is worth the headache.  Some of these will need ISBNs, and the pricing and royalties will all vary of course.

Printed Books


If you've spent a lot of time making a great book, you probably want to see it in print format, right?  Nothing is quite the same to an author as being asked to sign a physical copy of professionally bound book of their own work.  Note that it takes some extra effort to set up a book for print -- you'll need to ensure that you have a press-ready PDF (I had to purchase a copy of Adobe Acrobat DC so that I could properly preflight my files), and setting up the cover can be a challenge if you're not a designer.

Note that details such as print margins, paper weight, and hence cover sizes, can vary between the different printers.  Be prepared to spend a lot of time if you decide to go down this road, and to have to spend a lot of time for each printer you use.

TheBookPatch.com


After doing some research, I decided to give these guys a shot at printing my first version.  I was really impressed with the quality -- while the first printing of my book had a number of issues, none of them were the fault of TheBookPatch -- they were all on me.  The problem with these guys is that they are tiny.  Almost nobody has ever heard of them, and you won't get be getting this listed at places like Barnes&Noble.  Additionally, they are rather expensive, particularly if you want to send books to places overseas.  At one point I wanted to send one copy of my book to the Netherlands.  The shipping cost was going to be about $80.  Needless to say, my relationship with TheBookPatch came to an abrupt end.  (I'd still recommend giving these guys a shot if you're printing books for your own use here in the USA.)   One big advantage is that they were able to put together an attractive cover using their website cover designer, with no special skills.  You also don't need an ISBN to print through TheBookPatch.com.

Ingram Spark


Ingram Spark has the best rates internationally, and is reputed to have excellent print quality.  My book is available from them.  They charge $49 to set it up, and $25 for updates.  This is super annoying, so I wouldn't publish with them until and unless you know that you're ready and need international distribution or want to see your printed book available via Barnes&Noble or other retailers.  They're also slow.  I ordered 3 copies of my book a week ago, and they only confirmed that they are shipping them today.  If you're serious about selling printed books widely, I would definitely go with them.  But unless you anticipate the volume, I'd hold off.  You will need an ISBN as well.  With Ingram Spark, you set up your royalty rates which are usually 45% of net.   Typically this means you'll get something like a 20-25% actual royalty, depending on the book.

Amazon KDP


Now available for authors, you can use Amazon Print on Demand.  After setting up the layout, and doing the work to ensure the quality is good -- which can take some effort -- it's pretty easy.  Amazon will sell you an ISBN if you want one -- I'm not sure if they are required for print books or not.  (I already had one from my Ingram Spark journey.)  Amazon gives a much better royalty, of 60% of net, and their printing costs for small runs seem to be fairly inexpensive, as is shipping.  For example, my 430 page, 2 lb (7.5"x9.25" paperback) book cost about $6 to print, and about $10 to ship.  That means that as my list price is $49.95, I can expect to receive about $20.  Amazon will cut into their own margins to discount the book as well, to optimize the price.  Having said all that, I'm still waiting for my proof, which Amazon apologized for taking an extra day or two to print -- I should be getting it in a couple of days (I opted for the cheap shipping -- you can't use your Prime account to ship author proofs which are made available to you at cost).  Their paper is thicker than Ingram's and so I had to redesign the cover, and their margins are stricter (my page numbers fell outside their strict .5" margins), so I wound up having to re-do the whole layout.  It would have been better if I had started with Amazon first.

There are other print-on-demand players, but I've heard enough complaints about print quality when using them, that I just avoided them.  After all, if you're bothering to put your book into print, you want the results to reflect all the effort you put into it.

2018-06-05 Josef "Jeff" Sipek

Smart Clock: A New Time — Using three inexpensive wrist watches to achieve 1 second accuracy over an extended period of time.

Repairing the card reader for a 1960s mainframe: cams, relays and a clutch

The 555 Timer IC an Interview with Hans Camenzind—The Designer of the Most Successful Integrated Circuit Ever Developed

High-level Problems with Git and How to Fix Them — A Mercurial developer’s view of Git’s shortcomings.

Mailing lists vs Github

GDL 90 Data Interface Specification — Definition of the serial protocol used by Wikipedia article: UAT receivers to feed the received data to Wikipedia article: MFDs.

GDL 90 Extended SpecificationForeFlight’s extension to GDL 90.

Altering the deal... again.... /dev/dump

(No, this is not about GitHub or Microsoft... lol.)

Back in March (just a few months ago), I signed up on Leanpub to publish the NNG Reference Manual.  I was completely in the dark about how to go about self-publishing a book, and a community member pointed me at Leanpub.

Leanpub charged $99 to set up, back in March, and offered a 90% (minus 50 cents) royalty rate.  On top of it they let me choose a price from free, or $0.99 to $99.  Buyers could choose within that range.  This looked great, although I was a bit hesitant to spend the $99 since there was no way to try their platform out.

Note that at this time I was not interested (and am still not interested) in their authoring tools based on Markua.  I had excellent tooling already in Asciidoctor, plus a bunch of home-grown tools (that I've since further expanded upon) to markup and layout the book, plus previewing, etc.

Everything was great, and I made sales ranging from $0.99 to $20.  Not a lot of sales, but enough to nearly recoup my $99 investment.  Now, I wasn't looking at this as a money making venture, but as a way to help support my work around NNG -- having a professionally produced reference manual was something I considered an important step for NNG.

Shortly after I created the book and published, Leanpub changed the minimum price that buyers could pay to $4.99.  We're talking about a digital good here.  First time the deal was altered....

Then in April, they introduced a new SaaS pricing model, where I could have ditched the $99 fee.  So I'm feeling like a chump, but hey at least I have that 90% royalty rate, right?  (By this time I'd sold enough to cover that initial $99 outlay, thanks to generous supporters from the NNG community.) . Deal altered again.

Then they introduced a freemium model in May, where I really could have skipped that $99 outlay.  But they told me that I was grandfathered, so I could keep my 90% rate, so I was getting something for that $99 I spent originally.  Deal altered third time?

Now, they've told me that they've changed their mind, and no, they aren't going to let me keep that grandfathered rate.  Deal altered again?!?

They posted a long essay explaining why they "had" to do this.  I get it, their old business model wasn't working.  But in the past 3 months they've made not one, not two, but three changes to their pricing and business model.  They've made promises, and gone back on their word.

But it's ok, because at 80% I'm making more than with Amazon, right?  Well, no, not really.  I won't repeat the calculations here, but it turns out that I would have made slightly more money with Amazon.  Now, that's partly due to the fact that my sales have been quite slow (as they were predicted to be -- this is a really niche book -- a reference manual for a product that isn't even 1.0 yet.)

The thing is, I'm slightly irked about the loss of income, but I'm much more angry about the lack of respect they've given us, their authors and customers.  Clearly, their promises don't carry much weight.  They've offered lifetime free Pro accounts to customers who were with them long enough to have at least $500 in royalties, but everyone else is out of luck.  As to those lifetime pro accounts -- well, it's "lifetime, or until we change our mind".   Which seems to occur about once a month.

Now Leanpub isn't some big bad company, but their attitude and thinking reflected in how they've handled this process shows clear alignment with the same thought processes that those big bad companies have.  As an author you're not a valued partner to them -- you're a source of revenue, with very little effort on their part required to support you.

I've started rethinking my use of Leanpub obviously.

It seems like I can make use of Selz which seems to have really good support for selling digital goods like eBooks (and even has a Pay What You Want option!), and with my small number of digital goods will only charge me the transaction processing costs -- either 2.9% or 3.9% depending on location.  (Digital goods are not taxable in California.)  So what was I gaining from Leanpub again?

For Kindle and iBooks, it also looks like dealing with Amazon and Apple directly look like a better deal than Leanpub.  You get their expanded distribution, and yes, you only get 70% royalties, but you don't have to pay any recurring fees.  Unless you're doing large volumes, the math on these works out better than any of the Leanpub paid plans.

 (IngramSpark, where I have also posted the book, also works, but I've had less than satisfactory results with their epub->mobi conversion, so I can't recommend using them for Kindle at least, and I think the royalties you get from dealing directly with Apple are superior anyway.)

This all seems like a lot of work, but I hope this helps other authors who might be considering using Leanpub.

(There is one feature which is nice on Leanpub, which is the ability to publish an incomplete work in progress, and then keep updating it.  But let's face it, you can do that equally well from your own website and something like Selz.)

Not Abandoning GitHub *yet* /dev/dump

The developer crowds are swarming off of GitHub in the wake of today's announcement that Microsoft has agreed to purchase GH for $7.5B.


I've already written why I think this acquisition is good for neither GitHub nor Microsoft.  I don't think it's good for anyone else either... but maybe at least it alerts us all to the dangers of having all our eggs in the same basket.

At the moment my repositories will not be moving.  The reason for this is quite simple -- while the masses race off of GitHub, desperate for another safe harbor, the panic that this has created is overwhelming alternative providers.  GitLab reported a 10X growth.  While this might be good for GitLab, its not good for people already on GitLab, as there was already quite a well understand performance concern around GitLab.com.

At least in the short term, GitHub's load will decrease (at least once all the code repo exports are done), I think. 

The other thing is that Microsoft has come out and made some pretty strong promises about not altering the GitHub premise, and the "new leadership" over there is ostensibly quite different from the old.  (Having said that, there is a lot of bad blood and history between FOSS and Microsoft. A lot of the current generation of millenials don't have that history, but some of us haven't forgotten when Steve Ballmer famously said "Linux is a cancer", and when Microsoft used every dirty trick in the book to try to kill all competitors, including open source software.  If Microsoft had had its way back in the 90s and 00s, the Internet would have been a company shanty-town, and Linus Torvalds would have been a refugee outlaw.

Thankfully that didn't happen.

Microsoft is trying to clean its image up, and maybe it is reformed now, but the thing we all have to remember is that Microsoft is beholden first, foremost, and exclusively to it's shareholders.  Rehabiliting it's image is critical to business success today, but at it's roots Microsoft still has those same obligations.)

The past couple of years of good behavior doesn't undo decades of rottenness; many of us would have been thrilled to see Microsoft enter chapter 11 as the just dessert for its prior actions.

Microsoft was losing mindshare to OSS and software like Git (and companies like GitHub). Purchasing GitHub is clearly an effort to become relevant again.   The real proof will be seen if Microsoft and GitHub are still as FOSS friendly in two years as they are today.  Promises made today are cheap.

But I'm willing to let them have the benefit of the doubt, understanding that I retain my options to depart at any time.  I won't be creating *new* repositories there, and my private one's will be moving off of GitHub because I don't want Microsoft to have access to my proprietary work.  (Probably they can still get it from backups at GitHub, but we do what we can...)

But my open source stuff is still there.  For now.

That means mangos, NNG, nanomsg, and tcell remain.  For now.

It's up to Microsoft and GitHub to see if they stay.

 - Garrett

Microsoft Buying GitHub Would be Bad /dev/dump

So apparently Microsoft wants to buy GitHub.

This is a huge mistake for both companies, and would be tragic for pretty much everyone involved.

GitHub has become the open source hosting site for code, and for a number of companies, it also hosts private repositories.  It's the place to be if you want your code to be found and used by other developers, and frankly, its so much of a de facto standard for this purpose that many tools and services work better with GitHub.

GitHub was founded on the back of Git, which was invented by Linus Torvalds to solve source code management woes for the Linux kernel. (Previously the kernel used an excellent tool called BitKeeper for this job, but some missteps by the owners of BitKeeper drove the Linux team away from it.  It looks like GitHub is making similar, albeit different, commercial missteps.)

Microsoft already has their own product, Visual Studio Team Services, which competes with GitHub, but which frankly appeals mostly to Microsoft's own developer base.  I don't think it is widely used by Linux developers for example.


Implications for Open Source


Microsoft has been much more "open source friendly" of late, but I have to admit I still don't trust them.  I'm hardly alone in this.

It also is a breach of sorts of an unwritten trust that the open source community has placed in them.  There is much bad blood between Microsoft and open source software.  Many of the most treasured open source systems exist directly in conflict to proprietary systems.  Think about software like Samba, and Wine and OpenOffice.  These were created as alternatives to Microsoft.  Being acquired by Microsoft means that these projects will feel compelled to abandon GitHub.

As this happens, many tools and services that offer services that are tailored to GitHub (automated code review, CI/CD, etc.) are going to be rushing to find a way to offer services for alternatives, as their client base runs screaming from GitHub.  (Back in February of 2016 I tried to leave GitHub, because of philosophical differences of opinion with their leadership.  I abandoned the effort after discovering that too many of the external support services I used for these open source projects were either GitHub only, or could only be converted away from GitHub with large amounts of additional effort and big negative impact for my users.)

This is a watershed moment for GitHub.
I predict in as little as 6 months nobody will be creating new open source projects on GitHub.

Unfortunately, it's probably already too late for GitHub.  Unless they were to come out and immediately deny any acquisition attempts, and make some public announcements recognizing the trust they've been given, and asserting the importance of honoring it, nobody will trust them any more.


Implications for Commercial Use


This is also going to harm commercial customers, driving them away.

Microsoft has many commercial ventures which overlap with those of almost everyone doing anything in software.  GitHub being acquired by Microsoft will in one fell swoop make GitHub a direct competitor with vast amounts of their own customer base.  (Essentially, your either a Microsoft competitor, or a partner.  And often both.)

If you're using GitHub for private repositories, it probably is time to rethink that.  Unless you trust Microsoft not to do evil.  (They've never even made any such promises.)  This also means, I think that it might be time to reconsider hosting your private data with anyone else.  GitLab and BitBucket look better to be sure, but what's to prevent another large company from acquiring them?

It's time to reconsider the cost of hosting in the cloud.  I've been expecting a move back to on-premises storage and hosting for some time now, but this will only accelerate that.


Implications for Microsoft


Microsoft will spend quite a lot of money to acquire GitHub.  But instead of acquiring a goose that lays golden eggs, they are going to have one that needs to be fed and turns that into fecal material.

At the same time, while this may help bolster some of the technology in VSTS in the short term, the reality is that most of the best stuff isn't that hard to build, and most of what GitHub has can be done on any cloud based system with sufficient storage and compute.  Most of their tech is not tied to Windows, almost certainly.

The VSTS team will no doubt be impacted, and there will be a lot of pain and suffering attempting to more tightly integrate VSTS with the new adopted child.  I'm sure there are redundancies that will be eliminated, but I expect part of what is going to happen is a shift in focus from providing the best experience for Visual Studio developers and making things work well on Azure, to figuring out how to more tightly integrate GitHub's toolset into theirs.  Can you imagine trying to reconcile the differences between VSTS and GitHub's issue tracking systems?  Yikes!

The uncertainty will annoy customers, and I suspect will drive them away from the existing VSTS stack.  Whey they leave, they probably won't be moving to GitHub.

Like the proverbial dog with the bone looking at his reflection in the water while on the bridge, instead of having one bone, Microsoft's greed will leave it with none (at least in this space.)

I'm sure that the founders and investors of GitHub will make a mint taking Microsoft's money.  Normally I'd applaud anyone with plans to part Microsoft from some of it's funds.  But this move is just plain bad business.


Anti-Trust Violations?


As I mentioned above, Microsoft has their own product, Visual Studio Team Services, which competes with GitHub.  This alleged acquisition of GitHub seems to me to fly in the face of anti-trust rules.  Microsoft clearly has been trying to make inroads into the open source community with projects like Visual Studio Code and Linux support for VSTS, so I would hope that the regulatory bodies involved would examine this with great scrutiny.

Of course, if GitHub is for sale, many of the same concerns except the antitrust legislation would apply.  It would be a Bad Thing (tm) if GitHub were to be acquired by Facebook, Google, or Amazon, for example, for most of the same reasons that being acquired by Microsoft would be bad.

Now please pardon me while I go back to setting up gogs on my own systems...

Tribblix - creating zones from images The Trouble with Tribbles...

One of the things I've done with Tribblix is try and hide some of the complexity around managing zones - rather than having to mess around with zonecfg and zoneadm and all that, just have one simple command that creates a zone correctly.

Tribblix also has the capability to drop an alternative illumos distribution into a zone - so called alien zones.

The OmniTribblix variant has LX zones, so you can run Linux in a zone.

Up to now, you've had to manually download the appropriate image, save it somewhere, and then install the zone from that image.

Wouldn't it be much easier to have Tribblix do that for you? Well, as of the m20.4 release, it can!

So, for example, OmniOS have images as zfs send streams available for download - the .zfs.bz2 files. So you can now build an OmniOS zone on Tribblix like so:

zap create-zone -z omnios -t alien -i 10.0.2.26 -I omnios:r151026

That's it.

And, if you're running OmniTribblix you could create an Ubuntu zone like so:

zap create-zone -z ubuntu -t lx -x 10.0.2.27 -I ubuntu

or, if you want Alpine (and this is really quick)

zap create-zone -z alpine -t lx -x 10.0.2.28 -I proxmox:alpine

The images are downloaded and cached, so creating subsequent zones will be much quicker.

It's a proof of concept at this point, and needs fleshing out a bit more to make it even more friendly, but it shows that it's possible and useful.

Linux bcc/eBPF tcpdrop Brendan Gregg's Blog

While debugging a production issue of kernel-based TCP packet drops, I remembered seeing a new function added in Linux 4.7 by Eric Dumazet (Google) called tcp\_drop(), which I can trace using kprobes and bcc/eBPF. This lets me fetch extra context to explain why these drops are happening. Eg:

# tcpdrop
TIME     PID    IP SADDR:SPORT          > DADDR:DPORT          STATE (FLAGS)
05:46:07 82093  4  10.74.40.245:50010   > 10.74.40.245:58484   ESTABLISHED (ACK)
	tcp_drop+0x1
	tcp_rcv_established+0x1d5
	tcp_v4_do_rcv+0x141
	tcp_v4_rcv+0x9b8
	ip_local_deliver_finish+0x9b
	ip_local_deliver+0x6f
	ip_rcv_finish+0x124
	ip_rcv+0x291
	__netif_receive_skb_core+0x554
	__netif_receive_skb+0x18
	process_backlog+0xba
	net_rx_action+0x265
	__softirqentry_text_start+0xf2
	irq_exit+0xb6
	xen_evtchn_do_upcall+0x30
	xen_hvm_callback_vector+0x1af

05:46:07 85153  4  10.74.40.245:50010   > 10.74.40.245:58446   ESTABLISHED (ACK)
	tcp_drop+0x1
	tcp_rcv_established+0x1d5
	tcp_v4_do_rcv+0x141
	tcp_v4_rcv+0x9b8
	ip_local_deliver_finish+0x9b
	ip_local_deliver+0x6f
	ip_rcv_finish+0x124
	ip_rcv+0x291
	__netif_receive_skb_core+0x554
	__netif_receive_skb+0x18
	process_backlog+0xba
	net_rx_action+0x265
	__softirqentry_text_start+0xf2
	irq_exit+0xb6
	xen_evtchn_do_upcall+0x30
	xen_hvm_callback_vector+0x1af

[...]
This is [tcpdrop], a new tool I've written for the open source [bcc] project. It shows source and destination packet details, as well as TCP session state (from the kernel), TCP flags (from the packet TCP header), and the kernel stack trace that led to this drop. That stack trace helps answer the why (you'll need to browse the code behind those functions to make sense of it). This is also information that's not on the wire, so you can never see this using packet sniffers (libpcap, tcpdump, etc). I can't help but highlight this small but significant change from Eric's patch ([tcp: increment sk_drops for dropped rx packets]):
@@ -6054,7 +6061,7 @@  int tcp_rcv_state_process(struct sock *sk, struct sk_buff *skb)
 
 	if (!queued) {
 discard:
-		__kfree_skb(skb);
+		tcp_drop(sk, skb);
 	}
 	return 0;
\_\_kfree\_skb() is called from many paths to free socket buffers, including routine codeapths. Tracing it was too noisy: you'd have your packet drop code paths lost among many normal ones. But with the new tcp\_drop() function, I can trace just the TCP drops. I've already suggested some enhancements to Eric today at netconf18, such as adding a "reason" argument somewhere that I can trace for a more human description of why the packet was dropped. Maybe tcp\_drop() should become a tracepoint too. Here's some more code worth mentioning, this time some eBPF/C from my tcpdrop tool:
[...]
    // pull in details from the packet headers and the sock struct
    u16 family = sk->__sk_common.skc_family;
    char state = sk->__sk_common.skc_state;
    u16 sport = 0, dport = 0;
    u8 tcpflags = 0;
    struct tcphdr *tcp = skb_to_tcphdr(skb);
    struct iphdr *ip = skb_to_iphdr(skb);
    bpf_probe_read(&sport, sizeof(sport), &tcp->source);
    bpf_probe_read(&dport, sizeof(dport), &tcp->dest);
    bpf_probe_read(&tcpflags, sizeof(tcpflags), &tcp_flag_byte(tcp));
    sport = ntohs(sport);
    dport = ntohs(dport);

    if (family == AF_INET) {
        struct ipv4_data_t data4 = {.pid = pid, .ip = 4};
        bpf_probe_read(&data4.saddr, sizeof(u32), &ip->saddr);
        bpf_probe_read(&data4.daddr, sizeof(u32), &ip->daddr);
        data4.dport = dport;
        data4.sport = sport;
        data4.state = state;
        data4.tcpflags = tcpflags;
        data4.stack_id = stack_traces.get_stackid(ctx, 0);
        ipv4_events.perf_submit(ctx, &data4, sizeof(data4));
[...]
My prior tcp tools in bcc have made do with struct sock members alone (eg, [tcplife]). But this time I needed packet info to see TCP flags, and the direction of the packet. So it's the first time I've accessed TCP and IP headers in eBPF. I added skb\_to\_tcphdr() and skb\_to\_iphdr() in tcpdrop to help with this, as well as a new tcp bcc library for later processing in Python. I'm sure this code will get reused (and improved) over time. [tcp: increment sk_drops for dropped rx packets]: https://patchwork.ozlabs.org/patch/604910/ [tcpdrop]: https://github.com/iovisor/bcc/pull/1790 [bcc]: https://github.com/iovisor/bcc [tcplife]: http://www.brendangregg.com/blog/2016-11-30/linux-bcc-tcplife.html