fallenpegasus: amazon (Default)
So I have a string that has a binary data structure in it, and I want to use Perl's unpack() on it

The binary structure is a 32bit int, followed by N 8 byte chunks. And I don't have quadwords in this build of Perl, so I cant treat them as 64bit ints.

And the following unpack string doesnt work: "L(a8)*".

What would the correct unpack string be?
fallenpegasus: amazon (Default)
I've just checked in a small update to the s3-tools

I've removed the dependency on the Perl package XML::Simple, and replaced it with calls to XML::LibXML, which will have already been loaded because Net::Amazon::S3 depends on it.

I really dislike doing what should be a minor update via CPAN, and having a cascading set of added dependencies cause CPAN to pull in the whole world. So I shall swim upstream, and remove unneeded dependencies.


The tarball can be had at http://fallenpegasus.com/code/s3-tools

The Mercurial repo is at http://hg.fallenpegasus.com/s3-tools
fallenpegasus: amazon (Default)
A quick-n-dirty bit of Perl I've meant to write for years now.

Give it a list of filenames that contain kindasorta RFC822ish text, and it will "touch" each file so that it's filesystem timestamp matches the "Date:" header.

use Date::Manip;
foreach $f (@ARGV) {
unless (open(F, $f)) { print STDERR "fail open $f: $!\n"; next; }
read(F, $t, 8192);
close F;
if ($t =~ m/\nDate: ([^\n]*)\n/) {
$d = $1;
$secs = UnixDate($d,"%s");
unless (utime($secs, $secs, $f))
{ print STDERR "fail utime $secs $f\n"; }
} else {
print STDERR "fail parse $f: No Date header\n";

fallenpegasus: amazon (Default)
I just installed the Sun JRE RPM. I can't help but notice that it installed into /usr/java/jre*/lib/zi/ a nearly complete timezone database in it's own java-ish format.

All modern UNIX machines already have a completely complete timezone database, called the Olson Zoneinfo database, usually kept in /usr/share/zoneinfo/. It's kept exactingly complete and correct by an experienced and committed volunteer effort. The file format is tight, compact, and very easy to parse.

There is no excuse for Sun to clutter up my filesystem with a stale duplicate, and I'm willing to bet that their zi database is derived from an older version of the zoneinfo db.

They don't even need to change their API or any existing programs. I presume they were smart enough that the tz data is accessed thru some sort of abstract class interface. Just have that interface read and parse the zoneinfo db as needed, instead of their own db.

This rant was primed by my noticing that there is a CPAN Perl module that commits the same sin. DateTime::TimeZone contains a predigested and Perl-ized stale copy of the Olson database.

There is no reason for this! DateTime::TimeZone should access and parse the Zoneinfo db on demand, as it's used. There is no reason to waste space on millions of machines, in every CPAN replica, and on the bandwidth between CPAN and usermachines storing and hauling that data around, when it's already locally present.

Do Python and Ruby commit this same sin?
fallenpegasus: amazon (Default)
Every time I upgrade a couple of Perl modules in the CPAN shell, it ends up sucking in another dozen new modules. What is "Purple" (ah, a Perl interface to libpurple. (And when and why did that get installed?)). And why is it now sucking in "DBD::SQLite" and "HTTP::Server::Simple"? And now "DBD::SQLite" is forcing the load of "DBI"...

Eventually the Perl community might as well admit it up front, if you do anything, you're going to load a majority of CPAN into your machine. They might as well just make the perl binary a bittorrent node for CPAN...

And so many of these modules fail their test suite, and emit basic warnings as complaints about using uninitialized variables. I'm glad that people are writing tests, I just wish they would make their code pass those tests! All that downloading and time spent running tests, tests failed, modules not loaded, wasted time, wasted transfer.
fallenpegasus: amazon (Default)
Last night, I pushed out a formal release of my S3 tools.

The big thing is that it's now a Perl module, what with an installer and namespace in CPAN and everything.

It has a Freshmeat page. You can grab the tarball here, or check it out of Mercurial here.

It should soon show up in CPAN as Net::Amazon::S3::Tools.
fallenpegasus: amazon (Default)
Things are always harder and take longer.

I have s3getacl working and documented.

 $ ./s3getacl example
# bucket: example
# owner: 5a1568e09392dad4b4ceb54f29f0a64d651a531350d6f720fbd2367eed995f08
$ ./s3getacl example/thingee
# bucket: example
# item: thingee
# owner: 5a1568e09392dad4b4ceb54f29f0a64d651a531350d6f720fbd2367eed995f08
$ _

It uses Perl's XML::Simple. I'm thinking I'm going to need a more sophisticated Perl XML module to write the next step, s3setacl. If I do, I'll probably go back and change s3getacl to use it too.

I'm going to push this all up to my mercerial reposatory soon, for other people to use.
fallenpegasus: amazon (Default)
So, there was a command line toolset for manipulating S3, called the Hanzoarchives hanzo-s3-tools. They were kind of iffy in the installation and documentation, but they worked, more or less. However, their handling of S3 ACLs was pretty poor, very not useful. And as the last straw, it seems to have gone offline and vanished.

On the other hand, there is a really good Perl library, Net::Amazon::S3. Many of my clients are more or less Perl shops, so I have been recommending that they install it. But that module lacks the scripts that bring functionality out to the command line.

So I'm writing them. Today, I've got s3mkbucket and s3rmbucket working, complete with handling of ACLs and TLS. Tomorrow will be s3putacl and s3getacl, and quickly following will be s3put, s3get, s3head, s3rm and s3ls.

All open source, of course. It will be available in my public repository, and I'll see if I can get the Net::Amazon::S3 to accept it as a patch. If he doesn't, I'll just make a Freshmeat project, and advertise it that way.


fallenpegasus: amazon (Default)
Mark Atwood

September 2017

1011121314 1516


RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Sep. 26th, 2017 03:44 am
Powered by Dreamwidth Studios