A large portion of the time spent uploading a large directory with utilities like scp and sftp (and sometimes rsync) is sending all the meta information over for the various files (and creating them). For some reason, this sometimes takes a purely silly amount of time.

The upside is there’s a simple trick that will let you upload things very quickly. The tar command is optimized for dealing with lots of little files and does it far, far better than the other commands. It’s not network enabled, but ssh does understand pipes…

tar cz MyDirectoryOfLittleFiles | ssh remote.host "tar xz"

Ta da! Full-speed uploads with tar on either end doing all the file work and ssh just encrypting the stream. For me, over local connections, this gives a much more even upload experience than scp alone ever did, especially for large directories of small files (source, for example).

Downside: it doesn’t do syncing like rsync. If this fails in the middle, you get to start over or switch to rsync to finish it.

So I’m reading in a CSV file on the iPhone. I figure it’ll be good to link up the column title with the values by returning an array of dictionaries, each keyed to the column title. Then, I’m going to save that data into a Core Data SQL store so I can get it out of memory. Fantastic, yes?

Should have worked. Apple broke it.

Two things broke it, actually. While the strings that make up the column names are immutable strings that could have been simply retained as the dictionary key, Apple chose to copy them instead. When you have over a thousand records, that adds up fast. On the phone, it adds up faster. In fact, it adds up to about 1MB of memory wasted copying objects that cannot, by their nature, change.

The second failure happens during Core Data’s save phase. The SQL store now proceeds to copy all the values in each entity before saving them, keeping them all around in an autorelease pool that is flushed after save. That adds another 2MB to the memory usage right there.

So we go from a 500KB app heap at startup to 1MB after loading the string that represents the CSV file to 2.3MB after creating the initial array of data and then to 3.2MB of data after transitioning to the dictionary (same data, all unchanged and immutable objects). Then to top it off, Core Data brings the usage up to 4.5MB just to save everything. When it’s all said and done the heap goes down to 1MB.

So, yes, now I get to go through and create autorelease pools and flush them to capture objects behind-the-scenes and then ensure all my objects are flushed before telling Core Data to save, but none of this would be needed if Apple were simply smarter about the fact that their frameworks would be used in a low-memory environment that prefers to pass around immutable objects rather than making whole copies of them.

Copy bad. Retain good.

All the time I’ve used Cocoa, I’ve had one design pattern for creating an autoreleased object:

Guess what? You can twiddle -init and -autorelease.

Why would you want to? Well, consider:

Now, make it prettier:

See? There’s no dangling -autorelease on the end. Happy days!

Update: Class clusters break with this method, by the way. They’ll need the old [[foo alloc] init] workup.

If you create a model class in Django and then create a model subclass of that, then Django creates a 1:1 relationship to a table with the extra fields defined in the subclass. This results in there being an instance of the parent class as well as an “extended version” of that object in the subclass. That works out rather well for most cases, but there’s a few edge cases where it’s not sufficient.

Say I have an object A and a subclass B:

class A(Model):
	created = DateTimeField(auto_now_add=True)


	
class B(Model):
	modified = DateTimeField(auto_now=True)

I’ll wind up with some tables like this:

A

B

What this means is that if I instantiate PK 1 via A’s manager, I’ll get an A and only ever an A because the information about what kind of object it is exists only in B’s table. So if I have, say, a blog engine like Tumblr where there are text, image, video, audio, and other kinds of posts and I’ve made them all model classes based off RootBlog, I can’t do a lookup by PK and expect the right object; I’ll always get a RootBlog object.

Traditionally, if you had this problem, you’d use abstract classes and just have a different object series for each kind of post. That works, but it means you can’t do shortcuts that just pass around an ID and get the right object — you have to pass around the type as well.

However, there is one trick that’s a result of the 1:1 relationship that could be of some assistance here. When you create an instance of our example object A, it will have a property called ‘b’ that is a link to its corresponding full instance of B. The problem is that we can’t easily follow this without giving the ancestor direct knowledge of all its descendants’ names.

• Never leave the machine plugged in all the time. Laptops are meant to be portable. Using it as a desktop that never runs on the battery will destroy your battery life.
• Cycles are your friend. Never letting the battery complete a cycle will greatly diminish your run-time. Try to avoid charging the battery unless it’s drained past 30%. Any time the battery drains past 50% and charges more than 50% counts as a cycle. The farther you let it drain before the charge – the better its overall health will remain.
• 30 cycles in a year is not a good thing.
• Let the battery drain completely a few times a week.

• Never let it sit for long periods of time without use. Batteries need to be loved or else they won’t love you.

And if you follow those “tips” you’ll have a very healthy NiCad battery and a dead LiIon battery. What is this tool thinking? Has he never looked into this at all?

CYCLES ARE BAD.

Leave the unit plugged in, don’t discharge if possible, and charge as soon as you can when you get near power.

Even Apple says this, so why is this ‘tard being an smug idiot?

One more reason not to trust “community” tips.

I watched the president’s speech on education just now. Fantastic speech, very uplifting, and totally devoid of political context.

Can we now agree that Glen Beck is a frigtard that’s just striking up fear and loathing with a soothing tone?

Anyway, I did watch it via the FB Live stream so I could see the stream of idiots’ comments flowing on the side as I watched. It was … enlightening. Here’re some golden ones:

Curtis Dale Johnson this a bunch of horse crap why would we won our children to listen to this its taking our children learning time???????? why.
10:53am – Denver, CO

I … yes. Exactly. Thank you.

Michelle Thornton They told me to pull my kid out of school to watch at home if I wanted my kid to see it!
10:56am
Mandi Foust Wofford is keeping her son home today so he can watch with me…we live in Texas…
10:57am – Dallas / Fort Worth, TX
Kelly Caplinger Silvers Getting ready to watch Obama’s speech to students. Very upset that they are not showing at my son’s school.
10:58am – Indianapolis, IN
Cody W Warner Shame on TX schools for not showing this.
11:09am
Faye Skaggs I can’t believe some schools aren’t even offering this to students…shame on you. This is bi-partisan and should be viewed as such. Good for you President Obama.
11:14am – Harrisburg, PA

I saw this one come up a lot. That was just sad to see.

Wesley Gaines HEY OBAMA SAND BLOWER GO BACK OVERSEASE.
11:07am
Wesley Gaines OBUMMER STAY THE HELL AWAY FROM MY KIDS AND THATS AN ORDER FROM A US MARINE.
11:10am
Wesley Gaines WHEN A MARINE TALKS OBUMMER U SHUT THE HELL UP.
11:11am

This fellow was rather prolific and I do hope he sees some black SUVs soon.

Haters

Pat Lowe thinks that the president shouldnt be addressing kids about school when they are in school. a bit hypocritical dontcha think?
_10:59am – Minneapolis / St.