fallenpegasus: amazon (Default)
I will be at the MySQL Conference in Santa Clara in a few weeks in the middle of April. This will be my fourth time there. Gear6 is going to be there as a sponsor, and some of my coworkers will be presenting, and I will probably share some stage time with them as they do.

My first time, was my first time. I was still working solo, as an independent. I met many MySQL-sphere people who are now personal friends, as well as being industry peers and contacts. I gave the talk that "put me on the map", for a project that was my opportunity to learn about distributed version control, cloud computing, Amazon Web Services, MySQL internals, and MySQL storage engines. I would look at the presentation schedule, and usually not even understand the titles.

My second year, I was working as part of MySQL Professional Services. My presentation was about MySQL User Defined Functions. The major buzz was "What will Sun do?". The technology arising that I was the most aware of was Memcached. One of the first memcached hackathons happened. The business thing I became aware of was Rackspace's gearing up to enter the cloud computing space. I learned later that the seeds of Drizzle DB germinated there. I would look at the presentation titles, and wish I could attend more presentations, because I could learn from almost all of them.

My third year, last year, I was working on Sun Cloud. The major buzz was "What will Oracle do?" and about the separate Percona conference and similar political/personal stuff. The tech buzz was about Drizzle DB. I would look at the presentation titles, and realize I could teach most of the presentations myself, and so focused mainly on networking and on the "hallway track".

It will be fun to see what this year brings.
fallenpegasus: amazon (Default)
Many people noticed my tweet just over a week ago that I tendered my resignation at Sun Microsystems. Many people asked me "what next?". Here is my What Next:

Today is the first day of my new job at Gear6, as the Director of Community Development.

In part, this means that I will be doing for Gear6 what Jay and then Dups used to do for MySQL.

I'm the face of my company into the Memcached / Gearman / Drizzle / etc communities, and just as importantly, I'm the face of those open source communities into my company.

I was very positively impressed with the people at Gear6 when we were interviewing each other. I truly believe that they want to do The Right Thing, both from an open source community member perspective, and from the perspective of the technology, their customers, and their investors.

Very soon, I'm going to be splitting my "blog existance" into several blogs for different aspects. My current existing LJ will remain for my personal stuff. There will be another one for professional / technical musing, and I will also create and manage one or more blogs just for Gear6's open source and technology topics.
fallenpegasus: amazon (Default)
My friend Gabriel came up with a working "progress bar" for ALTER TABLE. Until MySQL / Drizzle can do this "natively", this is a pretty neat trick:

http://gabrielcain.com/blog/2009/08/05/mysql-alter-table-and-how-to-observe-progress/
fallenpegasus: amazon (Default)
Drizzle. You can integrate it into your architecture.

MySQL / Postgres / Oracle. You have to integrate your architecture to fit them.
fallenpegasus: amazon (Default)
The Drizzle Developers,
hard at work



DSC_0129, originally uploaded by krow.



The Drizzle Developers,
hard at work,
thinking about Burning Man




DSC_0130, originally uploaded by krow.

fallenpegasus: amazon (Default)
Sunday morning, in the wee hours of the morning, while packing, I discovered that I didn't know where either my DL or my Passport is. After spending an hour searching, I gave up, and decided to throw myself on the mercy of the TSA. Which I did at the airport. The TSA supervisor checked my credit cards, FlyClear card, corporate ID card, and CostCo card, and then stamped my boarding pass. I only need to do this 3 more times on this trip, and then do a very deep search of my room when I get back.

After I landed at SJC, I discovered that Tim Lord has shared the flight with me. If I had known that ahead of time, I would have let him share my taxi ride from Capitol Hill neighborhood to the SEA airport. As it was, I let him share my ride to San Jose Convention Center. I was actually too early to check into my hotel room. So instead I checked in my luggage, and then went off to find the Postgres Day.

Meeting up with open source community geeks, watching lightning talks, hacking, taking pictures.

I accidentally left my jacket in one of the meeting rooms, and ended up having to bid for it in an auction. So now I owe $20 to the Postgres Foundation. :)


Monday, I spent hacking and geek socializing, hanging out in the speaker room. Some of the tutorials, but I didn't end up going to any of them. Lunch was at the Good Karma cafe, which I had stumbled across the last time I was in San Jose.

Also on Monday, I hooked up MontyT with someone who may have a solution for automating the build of Drizzle on Windows machines, without manually maintaining Visual Studio Project files, or porting Autotools to Windows.


Tuesday, I went to the Gearman tutorial, and while in it, started working on http://forge.gearman.org/ and also started implementing a bunch of basic "plumbing" gearman workers that need to exist, to link together filesystem, mogilfs, couchdb, amazon web services, memcached, and Erlang.

Right now, I'm sitting in the Ignite talks. BrianA just won an Google O'Reilly Open Source Award.
fallenpegasus: amazon (Default)
The fine folks at O'Reilly sent me reviewer copy of two books on Erlang



I am currently in the process of learning Erlang for a personal project. These books both measures up to the high expectations I have come to expect from Pragmatic Programmers Publishing and from O'Reilly Books.

Erlang is a difficult language to "sell", and is a challenge to learn.

Both books assume you have decently good programming skills, and don't need your hand held too much about the idea of programming, and instead show you how Erlang is different, it's unique and interesting features, and some of "how to think in Erlang".

Both are very good books for learning the language, and gaining basic skill in using it.

Both of the cover pretty much the same territory, in pretty much the same order. You only really would need one of them, but they are both equally good, so I can't recommend one over the other. Either get both, or pick one at random.


I wish there was more on "how to think in Erlang", especially since most programmer's intuitions about multiprocessing and concurrency, born of battle scars with multithreaded programming in C/C++, will be wrong.
fallenpegasus: amazon (Default)
While at the MySQL Conference last week, I stopped at the booth for HeidiSQL, and spoke with Bill Culp, who is the developer of jHeidi, about Drizzle.

A few days ago, he emailed me, saying that he had spent the time, and now jHeidi supports Drizzle.

So now we have a multiplatform GUI for monitoring, browsing, and querying a Drizzle database server.
fallenpegasus: amazon (Default)
I see three stages in the adoption and the evolution of the adoption of cloud computing:

1. Taking an existing application, and transferring it as is into a cloud provider. This is "forklifting".

2. Taking an application, and wrapping it up in cloud based provisioning and backup. This is the business space that RightScale and Zmanda help with.

3. Developing applications that implictly assume they are running in a cloud environment. I see a lot of talk of this, but not all that many running yet. But very soon they are going to be everywhere.
fallenpegasus: amazon (Default)
While sitting in [livejournal.com profile] krow's talk on Drizzle, I was hacking on my project to allow query logging to Gearman from Drizzle, and I found a couple of annoying bugs in GearmanD, and a critical bug in Drizzle. With MontyT's help, I found and fixed the Drizzle bug, and then showed the GearmanD bugs to EricD, who has gone off to fix them.

Maybe there will be an actual running demo of it all at my BOF this evening.
fallenpegasus: amazon (Default)
Two years ago, at the very first MySQL Conference that I attended, MySQL and IBM announced they were working a storage engine for IBM DB2.

When I went to work for MySQL PS, one of my first gigs was going to an IBM site and help them with that.

Now I'm sitting in this MySQL conference, watching the talk where the resulting storage engine is being publically described.

I feel some level of pride in my part creating this very odd duck.
fallenpegasus: amazon (Default)
Here I am again, at a conference away from home, waking up to learn that my employer has been acquired. MySQL to Sun. Sun to Oracle.

This time I have fewer opinions, and more things to think about.

I have many options and opportunties, and fortunately I am in a place where I can decide without rushing.

As for MySQL and Drizzle and Gearman and MemCacheD? They are all open source, they are all GPLed, so they are all remarkably immune to the threats and risks of corporate games. For as long as people use them, there will always be support and services and fixes and features for those projects.

Sun Java and JVM are open source and GPL now, so they also are safe.

For other stuff? I have no idea. I hope for the best.
fallenpegasus: amazon (Default)
There was recently an article in Infoworld, Slacker databases break all the old rules, that gave a rundown of the new non-transactional non-RDBMS database systems that are coming on the scene.

It contains a number of remarks about bank applications and their need for strict transactionally consistant RDBMS, like so:

The old guard who start reaching for their heart medication at the news of these new databases are usually bank programmers who want to make sure that the accounts balance at the end of the day.

None of them is right for everyone, and all of them are completely wrong for the bankers out there.

Last year, when I was doing the MySQL Professional Services gigs, I actually had a client who was a gigantic financial services company. I'm not going to name them here, and you've probably never heard of them, but the odds are pretty good that they manage some portion of your money, via two levels of outsourcing indirection. They manage quite literally hundreds of millions of accounts, with an combined value in the high 12 digits. Every account gets at least one change a month, often more, and it all has to balance to the penny. Every account they've ever had, and it's full transaction history back to the beginning of time, must be kept and maintained. And there is very very strict legal and accounting oversight, imposed by internal auditors, external auditors, their direct clients, and a whole alphabet soup of regulatory agencies.

And they do not use a transactional RDBMS. For many reasons:


  • They've been around so long that the term "legacy" has real meaning.
  • They have a LOT of specialized reporting applications, many of which have been signed off on and locked down by their auditors and regulators. It would cost effectively an infinite amount of money to port them over to using SQL.
  • Their dataset is so large, that it would take literally months to just copy it, let alone import it into a new RDBMs.
  • Part of the reason they are so successful in their field, is that their margins and costs are so low, that they beat all their competition purely on price, in a very price sensitive market.
  • Buying enough IBM DB2 or Oracle RAC systems would cost them more in license fees than they currently charge their clients.
  • Estimations of the scalable performance of DB2 or RAC indicate that it would take more than a day, with all the necessary interlocking transactional updates, to apply just one day's worth of updates.


But, in their entire history, they've never lost a penny, or an account. All without a traditional transactional RDBMS.

Instead they have a huge array of VMS ISAM datastores.

How do they do it? They run "transactions" intelligently, in their applications, piece-wise, and carefully.

Remember, keeping transactions in the DBMS is not a law handed down from On High. It's an assumption hack because it was assumed that programmers could not be trusted to do it in their applications, and could not be trusted to understand how to implement good data structures or good enough access patterns.

Often it's a useful assumption, just like assuming that programmers could not be trusted to handle instruction sets, direct jmps, or memory allocation, thus the rise of compilers, while loops, and garbage collection.

But sometimes you have to step back, change your assumptions, and add support for average programmers in some other way. Or make sure you get better than average programmers, and give them brain support some other way.
fallenpegasus: amazon (Default)
Working on Drizzle. A while ago the "log to gearman" module I wrote got incorporated into the main tree. The current task is to finish "get gearman via INFORMATION_SCHEMA", and then Drizzle will be able to push its query stream out into a chain of gearman based map reduce system.


So here is a question for the DBAs big system architects out there. If you had an array of database server instances, generating a compiled query log stream of many thousands of statements per second, and you could map, reduce, and analyze that query stream for "interesting stuff"...

What sorts of "interesting stuff" would you want to look for?


There is going to be a BOF on this at the MySQL Users Conference next week.
fallenpegasus: amazon (Default)
So it turns out that the answer to the question "when computers and television converge, what will we have" is "computers". link.

So the next question is "when computers and telephones converge, what will we have?"

I think the answer is going to also going to be "computers". Just very portable ones.
fallenpegasus: amazon (Default)
I just got back from the monthly MySQL Meetup. It's been moved from the Elysium to downtown, to the offices of Blue Gecko.

So the food wasn't as good, but it was easier to talk and listen.

The conversation orbited a lot around Drizzle, but also to practical issues of types of sharding, using VIEWs to access denormalized tables, data warehousing solutions, and using snapshots vs online backup tools for backups.

There was also a great deal of discussion about AWS; running databases in EC2 nodes, backing up stuff into S3, building scaling web services, and using CloudFront.

It was fun.


I was "on" most of the time; bright, cheerful, helpful, and outgoing.

Which means, of course, that as soon as I got home, I was exhausted. An introvert am I, still.
fallenpegasus: amazon (Default)
The PrimeBase PBMS storage engine is now running on Drizzle.

It uses blobcontainer plugin hooks that I was asked to add to Drizzle at the OpenSQL camp in Virginia a few months ago.

It's always very rewarding to see something I built enable someone else to do something cool
fallenpegasus: amazon (Default)
I'm trying to figure out for a friend what results in something like this

Restart the MySQL server and then run the query once. Then check the status variables.

Created_tmp_disk_tables = 0
Created_tmp_files = 362
Created_tmp_tables = 2
Sort_scan = 0
Sort_merge_passes = 0

What could be creating all those tmp files? It's not sorting on disk....

Profile

fallenpegasus: amazon (Default)
Mark Atwood

July 2017

S M T W T F S
      1
234567 8
9101112131415
16171819202122
23242526272829
3031     

Syndicate

RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Jul. 20th, 2017 08:30 pm
Powered by Dreamwidth Studios