Re: This was posted on Digg.com -- Linux Distribution blog/article

[Jeff Licquia <jeff@licquia.org>]

On Fri, 2005-12-09 at 13:59 -0500, Bryan J. Smith wrote:
> On Fri, 2005-12-09 at 12:59 -0500, Jeff Licquia wrote:
> > Interesting article.  I take it you like ports? :-)
> 
> That's funny, because if you didn't notice it, someone thought I was
> being anti-Gentoo in the comments.
> 
> As I said in my response, people look at my views how _they_ want them
> to.  I have been called anti-Gentoo/pro-Red Hat and pro-Gentoo/anti-Red
> Hat.

Sure.  But when you list a lot of criteria, and every one says
"Advantage: Ports", you tend to get the idea that the person in question
is a ports advocate.

Plus, your point seemed to be that integration testing is a) spotty even
in the package distros and b) redundant since you have to do your own
integration testing anyway.  Perhaps I was misinterpreting.

> > The impression I got from your article is that package-based distros
> > only have an advantage because of integration testing, and even there
> > only when it actually happens.
> 
> As an engineer, I consider regression and integration testing *HUGE*!
> So if a ports distro, such as Gentoo, puts that on me, I need to be
> prepared for it.
> 
> Furthermore, there is the argument whether or not well-tested community
> distros, such as Debian, are "not as good" as the "Enterprise" release
> that change even less.  I think Debian founder Murdock has the right
> answer on that with his Progeny endeavors, testing and configuration
> management is really a functionality built around your business.

You don't have to convince me that Progeny is on the right track,
especially since I'm responsible for that in a not-insignificant
way. :-)

> The "key" here is if the vendor's testing and fixed configuration
> interferes with your needs.  If you use everything Red Hat ships in
> RHEL, plus only certain, certified applications -- then RHEL is a major
> _value_!
> 
> If I maintain a web server with lots of added modules, and I regularly
> run into dependency and configuration hell, then it doesn't.  But that
> means the configuration management, testing and other details then fall
> on to me.  A common _mistake_ I see people do when they deploy Gentoo is
> just have each server fetch their own, and those changes are _not_
> tracked.  You _still_ need to handle fixed configuration management to
> ensure consistency, testing, security, etc...

Maybe I'm just ignorant on this point.  On Gentoo, how do you maintain
that fixed configuration, or reproduce it on a second system?  It was my
understanding that emerge just sucks down the latest.

> > Often, problems are caused not by what you link with at run-time, but by
> > what you build with at build-time;
> 
> Which is why the regression and integration testing falls upon _you_.
> If someone stupidly ignores proper configuration management when they
> deploy Gentoo, and have varying, differing configurations, they deserve
> what they get.

Right.  But a large part of that work is done for you when you use a
stable packaged distro.  In fact, so much of that work is done for you
that I fail to see how using ports could possibly be a savings for the
vast majority of needs.

> >  - Binary ABIs.  The blessing of only enabling the features you want can
> > also be a curse if you ever install software not provided by the core OS
> > that need features you didn't include.  Sure, you can rebuild those core
> > components with the new features, but that adds to the difficulty of
> > getting something installed.
> 
> Yep, that's the big kicker on Debian too, one that Progeny hopes to
> resolve with their forthcoming enterprise release that is compatible
> with RPM binary-only software.  The problem with Progeny's service model
> built around end-users knowing their needs and configuration management
> is the reality that not all software is source.  If it was, I actually
> prefer Progeny's approach to "enterprise" management over the "as a
> product/subscription" model.

Well, sorta.  We basically gave up on the end-user model years ago.  Our
customers are savvy enough to understand their needs better than the
average end user.

Plus, for binary-only software, the LSB is key to our efforts.
Binary-only software is OK for us as long as the standard they adhere to
isn't "RHEL 4".

But the biggest place we play is in the middle.  Most people who
customize care about very specific pieces of the puzzle.  They need to
diverge, and thus lose their vendor support, but they don't need to
diverge so much that they must do all their own QA on the whole distro,
or totally lose compatibility with the wider world.  Before, these
people were pretty much forced into the latter approach.  Now, they can
call us.

> > Further, I think you overstate the advantages of ports.  A lot of them
> > are contentious; for example, the relative merits of sparing features in
> > a world of 200GB disks and 1GB DIMMs, and the relative merits of
> > processor optimization on systems that are nearly always I/O-bound.  But
> > this comes to mind as the most interesting:
> > > Organizations who believe they can do a better, more relevant and
> > > custom drop of assembling, building and supporting Linux will see
> > > little value in an "enterprise" Linux.
> 
> Read that again -- only _not_ thinking I'm a Gentoo advocate (I'm not!
> I deploy RHEL/FC _heavily_!  And Debian more than Gentoo! ;)
> "organizations who _believe_ they can do ..." that means the
> organization is taking that responsibility upon themselves.  If they
> constantly find themselves "fighting" the limited flexibility of
> "enterprise" distros, they might be better directed towards putting in
> the regression and integration testing and configuration management to
> "roll their own" just for their specific applications.

I believe you about your use of Debian/RHEL/FC, and did before.  But I
still think this is an overstatement.

Ports start to become a good approach when the need diverges from
standard practice by more than 50% or so.  But I believe that the set of
organizations with those needs has a membership approaching zero.  Most
of the organizations in the distro business today would like to get out
of it, and stay in only because of the lack of alternatives.

> > This quote is demonstrably untrue.  Consider Sun JDS.  Sun certainly
> > knows how to put together an enterprise-grade Unix; why then did they
> > base JDS on SuSE?
> 
> It has to do more with their past mis-use of the Red Hat trademark.

That's not the question.  The question is: why did they use either RH or
SuSE, when they could have built their own?  Certainly no one thinks
they weren't competent enough to do so.

> > I expect that's because their enterprise-grade-Unix
> > engineers have better things to do.  And if Sun thinks using an existing
> > enterprise Linux is a better use of their time than assembling one of
> > their own, you can bet that 99.9% of the rest of the world will think
> > the same thing.
> 
> But who is Sun's customer?
> Who is Red Hat's customer?

The question here is not "who is X's customer?" but "who should X be
going after as a customer?"  If you ask me, Sun isn't quite sure yet in
the Linux space, and Red Hat has the wrong answer.

> Are they going after the large, enterprise systems support department
> that has their own resources to do all the customizing, regression and
> integration testing and configuration management on their own?
> 
> Or are they going after the customer who wants a fixed configuration,
> willing to pay for those updates, and possible tap their services?
> 
> It's not about "quality" -- it's about "who pays for it with what?"  Do
> I pay for it with my time and efforts instead of money?  I might
> consider that if my customization is unlike most others.  Or do I pay
> for the fixed configuration?  That's probably the right move if I'm
> running largely what is already supported -- and has been tested -- for
> thousands of others.
> 
> That's the "key"!  How much do you need to customize "off-the-path"?

But customization is not either/or.  Certainly JDS isn't just rebranded
SuSE, any more than Ubuntu is just rebranded Debian.

However, in both cases, they used a starting point, and keep divergence
as low as possible, rather than do all the work themselves.  Why?  Does
Sun fear that they can't hack maintenance of a Unix derivative?  Does
Canonical (who employs a boatload of top-notch Debian developers, and is
headed by a former Debian developer) fear maintaining a Debian
derivative?

In both cases, the reason they start with a known quantity is that it's
silly to throw away the integration work that's already happened without
a good reason.

Let me give you an example.  The functionality of hotplug has been
rolled into udev recently, but getting the right combination of hotplug
(or not-hotplug), udev, and kernel together isn't as easy as it might
seem, especially when designing an upgrade process.  How many
organizations really need to hash all this out for themselves, as
opposed to just taking whatever the vendor was able to figure out and
assume the vendor got it right?

Certainly, there are some who care about this.  But do they also want to
hash out the interaction between CUPS and GhostScript?  Or the right way
to compile the Java modules in OpenOffice.org 2?  Or whether to use
acx100 or ndiswrapper to support certain wireless cards?  I'd like to
see the organization that really has enough of an interest in all four
issues that they really need to customize it all, outside of OS vendors.
And that's just four out of thousands.

> > But even more important, having everyone build their own means that no
> > one can work together.  Enterprises don't buy RHEL to run Oracle on
> > because it minimizes their own integration testing; they buy RHEL to run
> > Oracle on because Oracle won't help them with problems they have if they
> > don't.  (Or SLES, but you get the point.)  And this is entirely
> > reasonable.  Why should Oracle support need to be experts in the
> > combinatorial explosion of Linux components?
> 
> Nope.  You buy enterprise for fixed configurations, testing by many for
> many -- certified against apps.
> 
> You use ports when you are highly customizing to the point where your
> needs and testing differs from others.  Then all that testing falls on
> you.  If you ignore it, you deserve what you get.
> 
> In between are the "rapidly changing" packages distros with sprawling
> repositories that try to strike a delicate balance.

I don't see that ports are actually being used in that way.  Most people
with radically different needs still start with a packaged distro and
customize that.

Trust me on this: the demand for support for a regression-tested Gentoo
in the market is so low as to be undetectable.  What would be the
difference between that and a package distro offering, besides the fact
that the ports system is a lot more difficult to reproduce reliably?

(This doesn't mean that demand for Gentoo itself is nonexistent, just
that Gentoo's penetration into the market for commercial customized
Linux is.)

Ports-based distros with real support interest are distros with such
high value that they overcome the ports weakness (OpenBSD), or distros
that mitigate the support burden with stable releases (again, OpenBSD)
or distros where the support interest comes from an OS vendor (FreeBSD).


-
To unsubscribe, send email to majordomo@luci.org with
"unsubscribe luci-discuss" in the body.