postgres@tardis:~$ createdb test
postgres@tardis:~$ psql test
psql (8.4.0)
Type "help" for help.

test=# CREATE TYPE rank AS ENUM ('general', 'sergeant', 'private');
CREATE TYPE
test=# CREATE TABLE military (id SERIAL PRIMARY KEY,
test(#   name VARCHAR(128),
test(#   rank rank);
NOTICE:  CREATE TABLE will create implicit sequence "military_id_seq" for serial column "military.id"
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "military_pkey" for table "military"
CREATE TABLE

Notice the use of SERIAL? That’s postgres’ AUTO_INCREMENT, basically. I like it better. Next, it’s time to make a text file with some data and compress it. Here’s what I put in the file (note that the spaces between the words are TAB characters):

general Lee
sergeant  Hartman
private Pyle

And compress it with gzip, making a nice little file:

hank@tardis:/tmp$ gzip file
hank@tardis:/tmp$ zcat file.gz
general	Lee
sergeant	Hartman
private	Pyle

Now let’s actually make a named pipe for postgres to read from:

hank@tardis:/tmp$ mkfifo namedpipe

Now that we have our named pipe, let’s start reading from it:

test=# COPY military (rank, name) FROM '/tmp/namedpipe' WITH DELIMITER E'\t';

The E'\t' part means to escape characters inside the single-quoted string, turning this into an actual tab character. All that we have to do now is use zcat:

hank@tardis:/tmp$ zcat file.gz > namedpipe

Immediately, there’s some output in the psql session:

COPY 3

So, postgres says it got 3 records successfully. Yay! Now, let’s display them:

test=# select * from military;
 id |  name   |   rank
----+---------+----------
  1 | Lee     | general
  2 | Hartman | sergeant
  3 | Pyle    | private
(3 rows)

So, this is a pretty good method to read in compressed files with postgres. I’ve seen many articles that use similar methods with postgres dump files, but it’s useful for bulk delimited data loading as well, as many times it’s prudent to compress bulk data files and not extract them before loading them. See the postgres COPY page for more information about this awesome function.

TR: What about the public-health benefits? Systems that house large quantities of patient data could enable new types of research studies.

KB: Absolutely, that’s something I get really excited about. It will totally break open our knowledge base. For example, I have been diagnosed with low-pressure glaucoma, which is fairly unusual. No one knows what causes it. I would love to be able to search the system for anyone with this form of glaucoma and start to look for similarities.

I’d love to be able to do that too, except it would potentially violate the privacy rights of all of those individuals if they hadn’t agreed to specifically let you see their records. If they were to elect to share their information to help others find similarities as she suggested, that would be fine, but we should not assume everyone will do that, and we would have to have a process for this election upon diagnosis.

The first issue to cover is whether the Internet will be used as the medium of record transfer, if point-to-point connections will be established using the phone system or another network, or if an entirely new network will be created to facilitate these transfers, like the financial network. This article assumes the first option, especially since citizens will supposedly have access to the information online.  A separate network would be a much better solution, but would cost much more to deploy.

Why does it matter?

What are the non-privacy-related implications of Internet-accessible health records vs. them being on paper in a drawer? Most of them have to do with hacking, bribery and blackmail. Let’s say someone pays the Database Administrator of the health system $1,000 to change your health records to indicate you saw the doctor about gonorrhea (or they simply use a stolen doctor’s account, or they’re a hacker, etc. etc.). Now, they give you a call, letting you know that they’ll tell your wife unless you pay $10,000. Blackmail is a huge possibility. This is possible now, but only by those who work with patient information physically.

One effect of the centralized hackable database of health records would be the illegal issuance of prescriptions for drugs like Valium, Oxycontin, etc. for a fee. All that would have to happen is a falsified entry into the database, and you can go down to the store and pick up your bottle. I don’t necessarily oppose loosening rules on prescription drugs, but creating a new electronic black market for health record falsification could prove dangerous. After considering this possibility further, it would be possible to remove prescriptions from the system as well, possibly endangering lives.

So we should just use paper?  Come on!

Now, I’m not necessarily against digitizing health records. If each citizen, on the initialization of their record, was given a private key, and all the records were encrypted with the matched public key, and kept in a large central database, that would be fine. Yet, there are problems with this too since in an emergency, the health records wouldn’t be accessible unless the patient was conscious and able to type their passphrase. Therefore, there would have to be an override of some sort, which would destroy the security of the system. This override could be a “health safety deposit box” provided to patients optionally by a private corporation, which would contain their passphrase for emergency use, and would be authorized for query by the living will of the patient. This is the only possible way I can see for centralized health records to be implemented securely, but it seems to be unworkable at the moment.

So what about decentralization, which is what we currently have with paper and with the SAFEHealth system. If the records were kept by the doctor, and encrypted with both his and the patient’s public keys (for patient confidentiality), that would be secure. Of course, assuming the medium of transfer is the Internet, the decryption and changes would have to be done on a standalone computer to prevent the cleartext from being retrievable from the Internet, and any transfer to another office would involve re-encrypting the files with the other doctor’s public key, transferring of the result to an Internet-enabled machine, and the reverse process on the other end to read the records. Because this is painful and time-consuming, doctors and administrative assistants (I like ’secretaries’ better, but whatever) would obviously skirt the security here. And human involvement to decrypt would still be needed in emergencies. I’ve just sent an email to SAFEHealth asking for more information about their system:

Hello,
I’m interested in learning more about how your system works at a deeply technical level. Could you please point me to an explanation of exactly how records are stored, accessed, encrypted, decrypted, which keys are used, who generates those keys, and what network protocols are used to access the information? Thanks.

The only workable solution seems to be the patient signing away the rights of the government to make his/her health records potentially public information. We’ve seen various scandals involving medical industry employees already, like Octomom. Many people would sign this, especially if they went to different doctors all the time. Many people don’t care about their medical records being public, so they’d do it for convenience. But at a process for removal from the system at any time should be available for all patients. A system like this might complicate seeking diagnosis for things like alcoholism, opiate addiction, and mental health for fear that one’s employer might find out about the condition. Any patient should be able to elect to use paper instead, and be responsible for the transfer of his/her medical records to medical professionals for treatment.

But it’s for your own good!

A dangerous assumption is that we must force the patient to allow doctors access to their medical records for his/her own good. The fourth amendment exists to prevent this very thing from happening. It could also be argued that random searches of homes would discover meth labs and would save children, but it is unacceptable in this country because of our natural right to privacy. One way to assure access in case of emergency for those who have privacy concerns is by using a living will to allow access to the paper records assuming they’re filed somewhere accessible. Private companies could provide medical record storage facilities for profit, and could be called in case of emergency need of the records (or as I described before, the passphrase to unlock the records).

If one thinks this article is scathing to the whole idea of digital health records, he/she should have a look at this one.  While some of the same concerns and many more are brought up, different fears are addressed.  The corruption of government employees would also be a danger (which I touched on with the DBA bribe example earlier), but some of the later examples (the police officer having access) are a little unfounded and paranoid.

Microsoft and Google both have products for storing large amounts of health information.  When stories like this are appearing all the time, that really concerns me.  I’ll finish up with a good quote from this article.

“Ironically, HIPAA creates felony penalties if a doctor or hospital abuses the data, but there’s absolutely no penalties for a Microsoft or a Google because they’re not covered by the law,” Brailer said. “It’s nothing that they’re doing wrong. It just shows you the state of mind of Congress when that rule was written 10 years ago, because they never ever envisioned there would be online services managing health information.

“I think that’s a very high priority, because one consequence of the President-elect ramping up people’s attention to this is that people will come back to a lot of their fundamental worries about the protection of their health information,” Brailer said.

I look forward to comments and suggestions for this post.  This is definitely a hot-button issue at the moment, and any constructive criticism will be appreciated, and probably responded to.

Update 4/10/2009

So, Lawrence Garber, Principal Investigator for SAFE Health, and I have had a great email thread going on the security details of their system.  It sounds pretty good, but there are still concerns.  Apparently, they use HTTPS over a VPN, which isn’t a bad solution for network traffic security.  Yet, the last response I received from him indicated the following:

There’s no need or requirement to encrypt the data on the server because it’s within our physically, password-protected, and firewall secured datacenter. However passwords and credentials are encrypted.

So, again, we’re back to the unhackable datacenter with unencrypted data idea, which, from personal experience, isn’t a good idea.

CouchDB

CouchDB is a distributed non-relational database written in Erlang. It is unique in that its main query interface is simply HTTP REST, and for every UPDATE, it simply creates a new version of the row. Additionally, you can request the entire history of a row very simply.

Hypertable

An open-source implementation of Google’s bigtable. Hypertable uses novel methods such as Bloom filters to significantly decrease query times, as well as smart messaging to distribute a database across many nodes. It is also non-relational.

Creating and supporting Free Software in Africa

A group of CS professors hailing from Africa have gotten together to create a community that fosters creativity and innovation from people in Africa. People in first-world countries can participate by acting as mentors, or directly contribute to the projects involved. Chisimba is an open-source MVC framework for rapid application development. I am very interested in contributing to this project.

LucidDB

I thought going in that this would be somehow in the same ballpark as Hypertable and CouchDB, but I was disappointed. Basically, they are using compression and some fairly neat indexing to speed up traditional database queries. The main problem is that they only have a Java API, which completely turned me off after 30 minutes. Before that, it seemed like they were getting some pretty promising results. If they add some more APIs in the future, this may be another one to take a look at.

A History of Failure

An awesome talk by Paul Fenwick from Australia, generally detailing failures in computer science and engineering going back into the 20th century and even back to Roman times. This was a wonderful presentation – he’s a really good speaker – and it poked a lot of fun at New Zealand.

All in all, I must say that this OSCON is much better than last year’s at least according to what I was looking for in the sessions. The exhibit hall is also very good this year – I’m pretty loaded down with swag at the moment.

I know someone who would have gotten a kick out of Temporally Quaquaversal Virtual Nanomachine Programming In Multiple Topologically Connected Quantum-Relativistic Parallel Timespaces…Made Easy! had they been here. He needs to come next year (you know who you are..)

Tonight, I also attended FOSCON 4: Cooking with Ruby. This was a spectacular event hosted by Cubespace. I have to say that the live coding competition was a great spectacle, and held everyone’s attention for hours. It was an epic battle between Symfony, Rails, Smalltalk/Seaside, and Drupal. The rankings ended up being the following:

1. Rails
2. Drupal
3. Symfony
4. Smalltalk/Seaside

The presentations were good as well for the most part (notes here). AND THEY HAD BEER! I had some of the best keg beer imaginable – I thought it would be crap like you usually get out of a keg, but this was real quality Northwestern hopped pale ale. My cup says Bridgeport Ales, so I’ll have to investigate. If anyone knows the exact beer that was available in the left-side keg tonight, I’d appreciate a comment. I also met some cool people, some of which are all into XMPP and ejabberd. I may have to check all of that out now…