What to do when your cable modem doesn’t work

I got a call from my sister about her internet connection dying.  Here’s what I do in this situation – my internet is cable modem (CM) and my ISP is comcast.

  1. Attempt a reset by the ISP.  Through remote reset, the ISP can re-initialize your router and this may help.  If they *can’t* remotely reset your router, that seems like an indication that they may need to physically send someone to your home, doesn’t it?
    1. Gory details: Cable modems function on a standard called DOCSIS.  You probably don’t hear about this or think about it much but periodically your ISP may start nagging you insisting you need to upgrade your cable modem to a newer model to get the best service.  This probably means there is a new DOCSIS standard out, or they are offering support for a new standard and unless you get a newer modem, they won’t be able to sell you more expensive service.  You’ll probably only find yourself in this situation if you choose to buy your own cable modem rather than leasing one from your ISP – and you should almost definitely do that.  Newegg offers a bunch of DOCSIS 3.0 Cable Modems for under $60 and that’s what you’d probably pay in lease fees after 1 year.  Plus, why pay Comcast for this?
  2. Attempt to reset it yourself through power cycle.  Simply unplug the CM, wait a minute, then plug it back in.
    1. Waiting a minute may or may not be a magic amount of time (depending on your ISP, your CM, or the service you’re getting from your ISP with your CM).  Go ahead and wait a full minute though.
  3. Attempt to get in to the configuration for the cable modem and figure out what’s wrong from the logs.
    1. If you lease your modem, then your ISP has probably locked this down and you won’t be able to reach it (so go back to 1 or 2).
    2. Otherwise, it might be http://192.168.100.1 – so give that a try.
    3. Or, it might be printed on the bottom of your CM so get a flashlight and go try to take a peek.
    4. Or, you can do a web search for your CM model and probably find its default address that way.
    5. Assuming you were able to reach your cable modem’s configuration page, as it boots, you should be able to see a few useful bits of data:
      1. Where, in the connection negotiation process, it is failing.  This may or may not help
      2. Signal strength information.  The two most interesting values will probably be signal-to-noise (should be ~32dB or higher) and power (should be between ~-10dBmV to +10dBmV).  If your numbers aren’t in those ranges, you probably have an issue with your line connection somewhere and a technician will have to come to the house / line to assess where the problem is
      3. Full logs.  If you’re hitting a problem, you’ll probably find your logs end when they hit that snag and you may be able to research based on that information.

I hope this is useful!

Comments

TVR 2013 Recap

tl;dr

I haven’t been running much but aimed for 9 “miles” (6:40 pace/1600, 1:40 pace/400) for the TVR 1hour classic.  I hit it (pretty precisely; to paraphrase Stephen Colbert: Niemeyer, it’s German, bitch) and just a little over.  Thanks to Greg Crowther for being the kind of visible, competent runner to do things like this and make shlubs like me try, too, and thanks especially to Katie for her support and Elizabeth for some day forgiving me for being the reason she spent an hour of her life before her 2 month birthday hanging around a track.  Maybe next year I can aim for 16,000?

This year I was fortunate enough to be invited to attend Greg Crowther’s run for the Titus Van Rijn 1-hour classic. I didn’t make it, but I was motivated enough to take part myself.  So with the road to hell being paved with good intentions, I waited until flag day to finally drag myself to the track at Lower Woodland Park, near Green Lake and I put in my hour.

I haven’t been training much (my first daughter was born just over 6 weeks before race day) and I initially thought “OK, I’ll aim for 30 minutes at an 8:00 pace and take it down to 7:30 and see how I’m feeling.” This would have been 7.75 “metric miles.” I’m no contender for anything in any local races but it turns out I’m not that out of shape, nor am I that unambitious, so instead I decided to just aim for 9 metric miles.

“What’s a metric mile?”  Most tracks are 400m and 4 laps on a track is 1600m.  A mile is slightly longer than this (1609m) but for this I set my goals laps – so “metric miles.”

So, flag day (the deadline by which people participating in TVR must complete their runs) rolled around and after work Katie agreed she would come with the baby and they would support me on the event. Fortunately, most people have better things to do on a Friday night than run around a track for an hour, so almost no one was there for my 36 laps.  It might make some sense to warm up a little for a distance this short, but I didn’t really, and after a quarter lap jog back and forth, I started my watch.

My target distance comes out to 6:40 pace per 1600 and 1:40 per lap, 0:50 per 200.  This is nice and easy to calculate and check splits many times throughout the hour.  I’ve only been on a track two or three times in the past 2 months, but my first quarter split was right on and basically all my miles were, too.

The first mile felt a little more challenging than it probably should have, but I started getting into a groove shortly after that.  I’d left an Ultimate Direction gel flask on one of the benches with a little water and one Gu in it and I grabbed that for a sip after I cleared mile 3.  Unsurprisingly, this was actually super refreshing.

The middle third was when things started to feel a little challenging.  Katie is the best support anyone could ask for, but Elizabeth was getting restless and on every lap at least 50m before I got to them, I could hear her starting to cry a little. You might think “the sound of my crying baby might really get a fire burning in me and help me knock out some solid splits!” but you would probably be wrong.  It makes you want to quit and want to help your baby.  At least that’s how it made me feel and how I bet it would make most parents feel.  So I asked if she could take the baby on a stroll (the vibration of our new – and excellent! – Bob Revolution knocks her out) and got a few laps where I could just focus a little more.

The remainder of the event was pretty typical. It was hard, it got harder, I disassociated a little, focused on my breathing a little, focused on my form a little, thought about the event happening at the same time in Iowa City for my cousin that I was missing, and before I knew it, I was in the 7th mile – the last third.  At this point I was pretty sure I would make it but still wanted to quit pretty badly.  All of my miles had been very close, but I was afraid I was losing a couple seconds in the aggregate (I was watching the splits and if I hit 6:40.8 for 6 miles, I would have almost 5 seconds to make up if I wanted to get my target). I grabbed my gel flask and drank the last two sips as I headed toward the home stretch (meaning “remaining 12 laps”).

Mile 7 was tough and 8 was tougher, but the weather was cooling.  As I started the last mile, I looked at my cumulative time on my watch: 53:26.  This means my last mile would need to be 6:34 which isn’t crazy at all, but definitely let me know that I would need to push to make it.  And I wanted to quit – blah blah blah the same things every runner says – however here it was a little different because this isn’t a race, there wasn’t (really) an audience (OK, Katie understands this a lot better than I and she would know if I blew it or not), and I haven’t been training at all. But I thought of a couple things: first, I was 90% of the way done and now is not the time to throw in the towel.  Joe Gray said something similar to that in a recent interview I read – how it’s not until things start getting challenging and you see how you deal with it that you really start to understand what you’re made of. And, also, my cousin who I felt I was – if only in some personal and ceremonial sense – running for.  So I plowed through, crossed the 9 mile line with time to spare, started to stop, heard Katie yell “Go all the way through!” and picked it up a little to run until the full hour was complete, adding 33 meters on to my target.

Ultimately, races are tests of mental and physical fitness and they shouldn’t be that different from workouts.  The line between race and workout is certainly a lot thinner with the TVR 1-hour run, but it adds a new, interesting dimension on all of that and I’m very glad I participated.  I’m also super happy I was able to pull off that time because it wasn’t clear that I should be able to, given my training the last months with the baby, so I feel great about that, too.  And of course I’m incredibly lucky to have a good support crew – including one of the runners I respect most.

The event doesn’t end on the track, though.  Afterwards, participants are encouraged to indulge in a black cherry soda so before the night was over I had to find, and drink, some of that, too.

Splits – Friday 6/14/2013, Lower Woodland Track, Green Lake, WA. Sunny+warmish at start (~7:20) to overcast+coolish by finish (~8:20PM)

  1. 6:39.70
  2. 6:41.70
  3. 6:39.86
  4. 6:39.38
  5. 6:41.14
  6. 6:39.40
  7. 6:45.26
  8. 6:40.49 (53:26.93)
  9. 6:33.98 (last 433m)

Comments (2)

Thanks for the socks

As I got off the bus near the Space Needle – I noticed I missed a call from my dad and noticed it was from his cell phone.  This is weird – it’s never been entirely clear that my dad is able to turn the phone on (the last time it showed a sign of life, I realized my sister was visiting and sending text messages from it).  I called back and greeted my dad:

“Hi – I missed your call this morning?”

“Oh – Patrick – you got my voicemail?”

“No – I just called right back…”

I didn’t make it to Iowa City this past December for Christmas.  Tickets started at about $800 and seemed to just keep going up – but I found a much more affordable fare after the holidays and I spent a cold January weekend in Iowa. I’d talked with my grandmother around Christmas and she was, as always, a delight to talk with, but I could tell she wasn’t feeling well.  She’d been up and down with some sickness and injuries during the year and I remember her telling me on one call “You know, Patrick, I’ve lived a good life.”  It’s really hard to hear someone say this.  It’s not really made easier when you know it’s true, but I suppose it might help.

When I visited in January the latest news was pretty promising.  I’m really a rube when it comes to human health, but it seemed her symptoms might be a sign of pneumonia-of-some-type or maybe-some-cancer and it looked like Marian might be feeling symptoms from the pneumonia (which had a less-bad prognosis).  A few hours and one call later, it was pretty clear it was the cancer.

I visited as much as I could over the weekend but it was pretty clear that entertaining company – though she was, again, as always, great company – wasn’t really easy on Marian. We had a nice dinner at Blackstone.  I came nattily dressed in LL Bean’s finest addition to their catalog for winter 2012, but the weekend really came and went quickly.  In the 6 weeks since then, the news has been pretty clear and pointing in the same direction: her health was getting worse and pain increasing.

So it shouldn’t have come as any real surprise as I stopped while taking with my dad under the shade of a tree near Seattle Center where the homeless seek refuge from the rain.

“Marian died this morning.”

But of course it was a surprise, and awful, and I stood there sobbing in the rain and not knowing what else to do.

My dad put my mom on, who had been spending a lot of time with her mother these past months. My mom told me how Marian was still getting around and doing all sorts of things that I know would have made me think “You really shouldn’t be doing that…”  And I could only think that yes, she did live a good life.  I know she knew she was loved and she made her love known to the people that were important to her.

There are a lot of ways we measure our successes and failures in life, but really that’s about all that it comes down to, isn’t it?

IMAG1049

Comments

Programming a key for the Toyota Prius

Recently the key in my Prius has gotten really worn out. When I have asked my local Toyota dealer about getting a replacement key and getting it programmed for the car, I’ve been quoted a price of ~300-500 per key. Obviously, this is crazy, so I’ve been doing some research, ordered a replacement key off eBay, and found a number of things I wanted to collect more neatly for my own future reference or for others.

I’m not done with this process – I will update this post as I learn more or make more progress.

Dismantling the key and its main parts

The key fob consists of a few main parts – a good visual overview of this is on this site.  The entire block is referred to as the fob.  On the top of the fob is a small sliding lever.  If you slide this ~60% open, you can slide out a conventional metal key.  If you slide the lever all the way open and then apply pressure and push up on the back (the no-button side) of the fob, this will slide the case off of the fob.  From here you can see a sticker indicating your fob’s part number – mine is a B31EG.  There are multiple Prius key fobs for different car models, some of which have smart entry and start, some which just have the remote control keyless entry feature.  Below the sticker is a small, square plastic cover attached by four screws.  After removing these screws, you can replace a CR2032 watch battery that powers the fob for two of the key functions described below.  On the front of the fob (at least with my B31EG) the keyless entry buttons are surrounded by a sealing, rubber face.  This can be peeled back and I did – however I think it probably destroys your fob.  On mine, none of the buttons or keyless features were working already, anyway, so I had nothing to lose – but keep this in mind.

A thread on PriusChat.com indicates clarifies that there is no programming of the key, you program the car to accept the key.  A Prius will accept up to 5 keys to be paired with it (I don’t know how to purge an old key from the Prius). The Prius key fob has four functions and each of these functions is set independently.

The physical, metal key

As mentioned above there is a physical key embedded in the fob.  This is only used (on my 2004 Prius) to unlock the driver’s side door.  This is a physical key so there is no “programming” to do, but obviously it needs to be cut. Online advice recommends to have a locksmith cut it rather than simply going to a local hardware store.

The passive key that starts the car

This function allows you to turn on the car by placing the key in the car’s fob slot and pressing the Power button.  This works using some magic induction from the car (possibly passive RFID?) so if the battery in the key fob is dead, this will still work. I have successfully paired a replacement B31EG with my Prius by following instructions in this thread on priuschat.  This was pretty easy.

  1. [in/out*4] Insert old fob into slot and remove it 4 times in a row, then insert it a fifth time and leave it in.
  2. [open/close*6] Open and close the driver door 6 times (leaving it closed)
  3. [out] Remove the old fob from slot – the car is now in programming mode.
  4. [insert new, wait] Insert the new fob and leave in the slot for a full minute. Watch the blinking red anti-theft light on the dashboard.
  5. When it stops flashing and goes dark, your new fob is all set. Remove the new fob and you’re done.

Keyless entry (the remote control buttons)

This function enables pressing the remote control unlock/lock/panic buttons on the car and it relies on power from the fob internal battery.  Instructions on this vary and seem like they may depend on the model of the fob, the model of the Prius, or both.  I have not yet successfully completed this and am working on refining these steps, but the longest set of steps say to do the following:

  1. [open] With the driver door unlocked and no fob in slot, open the driver door.
  2. [in/out/in/out] Insert and remove the old fob into slot twice within 5 seconds
  3. [close/open/close/open] Close and open driver door twice.
  4. [in/out] Insert and remove old fob once.
  5. [close/open/close/open] Close and open driver door twice again.
  6. [in/close] Insert old fob in slot and close door.
  7. [power/power/power] Without pressing the brake, press power button, wait a second, press it again, wait a second, press it a third time to turn car back off.
  8. [out] Remove old fob from slot. You’ll be in “add mode” at this point. The car will lock and unlock doors to verify it is in add mode. (the precise number of times the car locks/unlocks the door is model specific – but it should do it at least once)
  9. …from this point on, the steps are not clearly written and I don’t know what works – further editing is needed…
  10. Press both buttons on old/new fob simultaneously for 4 flashes of the red LED on the fob.
    1. For a silver logo fob, the timing does not matter, but it is important to hold BOTH the old and the new FOB buttons simultaneously (e.g. hold down the lock and unlock buttons of a FOB but not simultaneously do it for both FOBs holding down all 4 buttons at the same time. I did this only once so I am not 100% certain about the sequence, but this is what I think worked for me:
    2. first press down both buttons of the NEW FOB simultaneously, car responds with locking/unlocking TWICE (not sure this step is even needed as car responding with locking/unlocking TWICE implies this is not a programmed FOB yet for this car)
    3. then, say 1 second after the car stops locking/unlocking due to step above, press down both buttons of the OLD FOB simultaneously,
    4. then, say after 1 second, press the LOCK button of the NEW FOB. Now, IF SUCCESSFUL car responds with locking/unlocking ONCE implying this new fob is now programmed/paired with this car.
      (iv) As already mentioned in previous posts, you have 40 seconds to play around with the steps in (iii) above. I had to play around a bit since it was not clear to me if one should first use the NEW FOB or the OLD FOB in the sequence of simultaneously pressing both keys. What is clear is that one needs to use BOTH FOBs
  11.  To end this programming, open a door or put the fob in the slot.
  12. Even though I don’t think it matters, I actually tried the smart key features after I drove the car for a minute while having the new fob in the slot. The smart key functionality worked fine.
Additional steps from another site:
  1. [no key, driver open, driver unlocked] The vehicle should be in the following condition-A. The key is NOT inserted in the ignition, B. The driver’s door IS open, C. The driver’s door is UNLOCKED.
  2. [in/out] Insert the Key into the ignition switch and then pull it out.
  3. [lock/unlock * 5] Press the Master Door Lock Switch 5 times from Lock to Unlock.
  4. [close/open] Close the Driver’s door then open it.
  5. [lock/unlock * 5]. Repeat step #3.
  6. Now select the mode by inserting the key into the Ignition Switch and turning it to the “Run” or “On” position. The programming mode is determined by the amount of times you go from “key off” to the “key on” position and back before pulling the key out.
    1. 1 time is the “Add” mode. This is used only on some models and it allows you to add a remote to the already existing remotes. The ECU confirms this by locking and unlocking the door locks automatically after you remove the key.
    2. 2 times is the second mode which will erase all previously programmed remotes and allow you to program new ones. The ECU confirms this mode by locking and unlocking the door locks twice after you remove the key
    3. 3 times is the third mode which tells you how many remotes are already programmed to the ECU. It confirms this locking and unlocking the door locks the amount of times applicable to the remotes coded. If no remotes are programmed then the ECU locks and unlocks the door locks 5 times. The ECU will hold up to 4 remotes at any one time.
  7. Press the Lock and Unlock buttons on the remote simultaneously for 1.5 sec and then press either button by itself for 1 sec.
  8. The ECU will perform the Lock/Unlock automatically to confirm that the 1st remote is stored by the ECU. Repeat step 7 immediately with another remote and continue until all remotes are registered.
  9. Shut the driver’s door and try all remotes.

Smart entry and start

This is the function that enables all the “magic” functions of the car.  This lets you grasp the doorknobs and will unlock the car if you have the key with you, lets you press the rubber buttons to lock the doors from outside the car (and prevents those buttons from locking the key in the car), and lets you press the “Power” button to turn the car on without putting the fob into the slot.  I have not found instructions on how to program this.

Comments

How I make calls, starring Google Voice

I’ ve tried to explain this a couple times recently to some people and my system is a little complicated so I decided to describe how it all works. Rather than talk about all the pieces and how they fit together, I think it’s easier to understand as a story of what bits and pieces I incorporated and what made them compelling because I made this switch over the course of more than a year.

What is it?

Google voice is a free service from google (at http://voice.google.com) that can overtake, extend, and greatly improve your experience with your cell phone. When you sign up for an account, you will pick a new phone number – your google voice number – and when this is set up, you can use it for sending and receiving text messages and calls.  Additionally, there is a slick voicemail replacement system that you can point your phone to so that your voicemails are stored in and managed by Google Voice rather than using whatever your cell company offers.

Voicemail replacement

My first use of Google Voice was to use it as my phone’s voicemail system.  I think voicemail is a giant pain. 9 times out of 10, the result of listening to a voicemail is either “can you call me back so we can figure something out?” or “here is a short message that would have been much more easily consumed (by the call recipient) as a text message.”  There are a couple benefits Google Voice offered, and the switch was pretty painless:

  1. The app (for Android, but available for other phones) does speech-to-text dictation. Now I can read messages rather than needing to listen to them (usually much faster).
  2. The messages are archived online. Have you ever wanted to save a voicemail in your regular voicemail inbox? Humans are emotional animals and we build attachments to sentimental things, but boy does voicemail make this taxing. Trying to skip past “saved” messages or go back and listen to bits and pieces of messages is a giant pain. With Google Voice as your voicemail provider, you can search for these in your browser (or in the app on your phone), see individual messages and callers (no more 9-9 to skip to the end of the message and choosing to re-save, blah, blah, blah) and so on.
  3. The messages can be played via computer. This is often less intrusive to the day than spending picking up the phone, clicking “call voicemail,” waiting for the call to answer, and browsing to new messages using that terrible interface. This isn’t a giant win, but it eliminates some annoyances and it is useful.
Text messaging replacement
This took longer to decide to switch.  The ability to use this comes automatically when you get your Google Voice account, though. In the same web UI where you see voicemail, there is a small, simple to use form for text messaging.  I tried this a few times but it’s a little weird for recipients to initially start receiving messages from you from an unrecognized number. Eventually I stopped caring about that in favor of the nice features it adds:
  1. Ability to type text messages on a computer keyboard – this is way faster and more convenient than tapping them on a phone.
  2. Automatic archive of all text messages – you might not care, but having this archived and searchable is really convenient.
  3. Ability to resume a text message conversation from multiple clients – whether it’s your computer, phone, or another internet connected device (Android or iOS), you have access to your whole text messaging history. This can be really nice.
  4. And, potentially, texting is now part of your data plan – you don’t need a separate texting plan (or you can downgrade to a cheaper one)
The only exception is for MMS (image / video) messages – Google Voice can’t send these, so recipients still receive this type of messages from the number assigned by your phone carrier, not your google voice number.
Number switchover for calling
This is the only thing that’s really left: broadcasting your Google Voice as your official phone number for incoming and outgoing calls.  This effectively “changes” your phone number on people (though your old phone number still works).  Here’s why this is compelling:
  1. You can set up the Google Voice number to proxy to multiple of your phones. When someone calls my Google voice number, three phones ring: my cell phone, my work phone (during work hours), and a home phone that I have.
  2. You can take the call on whatever phone is most convenient. If I’m at home, I prefer to pick up my home phone. This phone is comfortable to hold, has good call quality, and it won’t chew up my cell minutes. If I’m at work, I prefer to answer on my work phone for the same reason and also that my carrier doesn’t have great coverage at my work. Otherwise I use my cell phone.
  3. You can hand off the call to another phone, if it’s more convenient. I’ll often make or receive a call at home from my home phone, need to take my dog for a walk, and just pressed “*” during the call. This makes my other google voice phones ring so I pick up my cell phone (which was just conferenced in to the call), hang up my home phone, and now I can go for that walk.

So what’s the setup?

  1. On my cell phone, I configured it to change the phone number used for voicemail to the Google Voice voicemail number. This makes all the voicemail stuff work.
  2. On my cell phone and an internet connected device, I have the Google Voice app installed – this lets me send and receive text messages on any of these (as well as in the browser).
  3. On the Google Voice website, I have configured my google voice number to make all three of my phones ring – I also have a rule set up so that calls are not forwarded to my work phone during off hours when I am not at work.
  4. One more thing – my home phone is not connected to any phone providers at all.  I have an an OBi100 VOIP system plugged into my internet connection and configured to interact with my Google Voice account.  This was surprisingly simple and (until / unless Google start charging for Google Voice), all my home calls are free and go over the internet.

In closing

Google Voice is awesome.  It does a lot, you don’t need to adopt all of it at once, and everything it does, it does really well (to date).

I should also mention internet calling: this is possible, but the results are touch and go. I took a trip to Beijing earlier this year and I used my Google Voice number with another, great Android app called GrooveIP to make VOIP calls. This worked amazingly well. Almost no lag, as long as I had a reliable internet connection, I could comfortably make reliable calls. The call quality was actually better than I remember regular overseas telephone calls in the early 90’s. However, when I returned to the US and tried using this, I found >1 second latency in the call (this is well beyond “noticeable”). So the promise of this seems a little ways off, still, but I think it will get here..

Comments (2)

Possibly final notes on magazine scanning

I spent some time this past weekend experimenting with scanning settings and eventually got one full year of Northwest Runner magazine scanned. I chose 2003 to scan because I think this is a year for which digital copies exist – this meant if something went horribly wrong and I physically ruined an issue or two, I probably wouldn’t get my kneecaps bashed in at an upcoming Winter Grand Prix race series. I had already done extensive experimenting with my own copies of the magazine anyway, so that wasn’t too likely, but I wanted to play it safe.

Here are the key notes:

  1. Scanning in greyscsale images+text at 300DPI is dramatically faster than scanning in color.  However for recent issues this isn’t a great option. Some pages are B&W but many are full color.
  2. Output image size for greyscale vs. color at 300DPI is pretty similar.  The only reason this might matter is because of the behavior I previously mentioned where the only way I can do this with our scanners at work is to have it email the output from the scanner to me – I can’t scan directly to a network share – and my mailserver rejects messages above a certain size.
  3. My mailserver seems to reject messages when they cross a threshold somewhere between ~12-15MB in size.  In practice, this means I can scan about 5 ledger-sized, double-sided sheets, or about 10 pages of the magazine at a time.
  4. It is important to separate the pages and invert the fold along the spine before sending through the auto-feeder.  I didn’t do this with one of the first magazines and I wound up with some paper jams, some slightly mangled pages (not really destroyed or anything, but like what you get with a printer auto-feed after something’s gotten jammed).  I mentioned I scanned the entire year of 2003 – the jams only happened in the first issue or two.  After I started this separating and fold inverting process, the pages did not get “stuck” along the spine and they all fed cleanly.
  5. Sometimes 5 sheets barely hits the “too big” threshold for scanning. If this happens, I need to do something like “scan 3, then scan 2.”  This is rare, but it happens.
  6. Some of the magazines are missing pages or have single pages torn out.  This screws up pagination and might make later post-processing / assembly into PDFs a pain (or I might just ignore it).
  7. The printers at work require me to log in and after some time they will log me out.  If I’m logged out, I need to re-enter the scan settings (2 sided, color images + text, scan as JPG not PDF, 300DPI).  This is tedious.  If I stay attentive during the scan process, I can: feed 5 sheets, wait for it to scan, put the next 5 sheets in the feed reader, wait for confirmation that it sent the email, then press “Scan” again, I won’t get logged out.  This also ensures that the scanner (which is critical path in this assembly line) is always “busy.”
  8. As this process is happening, I’m getting email after email with 10 attached images (scan01.jpg, scan02.jpg, etc. for both sides) that I need to pull out of my inbox and archive in folders.  Because the image names conflict (scan01.jpg will be the cover and also page 11 and page 21, etc.) I need to batch these up, too.  My post-processing jpg rotater, cutter, etc. script will handle these.

That’s about it.  To scan the 2003 year of magazines took almost exactly 2 hours. During this time I am constantly busy with: de-stapling issues, preparing 5 page batches for the scanner (de-“sticking” the spine), running the scanner, adding/removing sheets from the feed tray, processing my inbox (which will fill up if I don’t pull the files out), reassembling scanned magazines, and trying to re-staple.  I think I can make this a little more efficient and bet I’ll trim a decent amount of time off that 2 hour baseline, but this process seems pretty close to optimized to do this job well and keep the original issues intact.

Now I just need to sync up with Martin (or really probably Bill Roe, who I think actually owns these issues) and confirm that they’re OK with me plowing ahead with all the back issues.

Comments

Next notes on scanning

After the initial research with scanning last week, I’ve concluded that trying to scan the back issues of Northwest Runner with my home scanners is probably a job I would never finish. At 1 minute per page and some non-trivial amount of post-processing (orienting all pages properly, assembling the PDFs), the initial time to scan is just more of my life than I’m willing to dedicate to this project. I found that my home printer/scanners do offer a document feed feature, though.  This works pretty well.  I can put a stack of documents in the feed tray, start the scanner function, indicating that the input documents are duplex with the moire suppression option on, click “go” and come back a half hour later, flip the scanned stack to get the other side and it’s pretty much done (and the postprocessing is slightly lower, too).

The problem with that is (for my printer) it requires an ~8 1/2 x 11″ input. I tried this with one of my own back issues of the magazine after taking the staples out and cutting it down the spine and the results were great!  Except for the original which I had cut in half.  This is no big deal to me, but apparently the guys who actually own these magazines I offered to scan and who have been involved in this sport for about as long as I’ve been alive are not exactly thrilled at the idea that I’ll destroy all their original issues.  Time for plan C…

This involves my scanners at work.  The printers at my work are Ricoh Aficio MP 5000’s and with *these* I can do scans of ledger (11×17) sized inputs, with the document feed feature, and will automatically do duplex scanning (no flipping required) and they are very fast.  These take about 6 minutes to scan an entire issue, front and back.  This leaves me with my last problem – how to get the scanned files from the printer.

It seems the Ricoh offers two functions – both of which present some problems.

  • Scan and send to email – this is kind of OK.  It will be inconvenient to need to pull the attachments out of hundreds of emails, but I could deal with it.  The larger problem is that the generated bulk scan from an entire issue is apparently larger than my mailserver will allow.  So I scan the entire issue over 6 minutes only to then have the printer tell me “sorry – couldn’t deliver your document” at which point those scans seem to be lost and I just wasted that time.
  • Scan and store on network share – this would be great except that the interface to get these things to talk with a Windows network share are maddeningly hard to use, might just not work, and might need some administrative rights with the printer that I don’t have.  After much trial and error with this, I think that this option is closed to me.

So my likely path forward will be to scan half an issue at a time (or so – if that’s possible) and go do post processing on those.  To do this, I will need to remove the staples from the back issues and feed in half an issue at a time, but I think it will work and go pretty quickly.  One thing I didn’t mention is that even with this approach, it *seems* that there are characteristics of the scan job that need to be re-entered every single time I start a scan job (select input as color, set DPI, other settings, original orientation settings).  Each of these is slow and tedious to input on the Ricoh touch screen and I’m hoping I can simplify it, but it might be tolerable and this will still be dramatically faster than working with my home scanner.

So – here are my next steps:

  1. Go back with a couple of my own copies of the magazine, do some trial and error to try to understand the maximum number of pages that can be scanned and emailed in one batch without my mailserver rejecting it and get more confident that the document feed will work smoothly / flawlessly before I send any of the originals through the feeder. This will include doing that for color inputs as well as greyscale (the oldest issues are greyscale, then a single color is added on some covers, then there are full color, glossy covers over greyscale pages, and current issues are glossy and full color from back to back).
  2. Start scanning the actual issues, probably starting from most current to oldest.  This way, again, if there are any problems with the process or I hurt some issues until I’m certain this is going perfectly, I have some time to correct the process.
  3. Start post-processing.  This may take a while.
    1. Probably use imagemagick, since I know some of its functionality
    2. Cut the scanned images in half – I’ll have ledger sized scans.
    3. Do some math to figure out page numbering.  If the cover is page 1 and an issue is 60 numbered pages long (back cover is page 60), I should have 15 input pages and 30 scanned images (front + back).  I think my picture batches will be: 1+60, 2+59, 3+58, etc.  Also, if I have to do this in two batches, I will have scanned images with numbering which will need to take some of this into account, too, in a cutting and renaming script e.g. postprocess [yyyy-mm] [first_page] [last_page].
    4. Probably also do some image rotation magic
    5. The final output of this will be perfectly named, oriented, and numbered scans (e.g. 1998-12-p01, 1998-12-p02, 1998-12-p03, etc.)
    6. I could deliver those back to Northwest Runner (this is what I had volunteered to do) or I might do some additional post-processing to attempt to assemble them into searchable PDFs.

And that should just about ruin my summer!

Comments (4)

Initial notes on scanning

So I’m starting to experiment with my scanning capabilities. I have two all-in-one printers. An old Canon MP530 and a newer Kodak ESP9250. I thought I would just use them both and cut my scanning time in half by swapping back and forth between them, but instead I’ve spent much of a lovely Memorial Day understanding their capabilities, what works, what doesn’t work and figuring out how I’ll actually scan 30 years of Northwest Runner magazine. Here’s what I’ve found.

First, the colors between the scanners is very different. Using the default scanning characteristics – here are some samples from the cover of the December 1998 issue.

CanonVsKodak

Obviously the picture is terrible – I’ll get to that in a minute – but the one on the left is the Canon and the one on the right the Kodak.  I ran a few more tests and the Canon gave me reliably more faithful looking scans of the original image than the Kodak, so I think I’m simply going to not use the Kodak.

Next – yeah, that image is terrible. How do I fix that?  That’s a moire pattern and it commonly happens with scanned images. The secret to fixing it is to set an option in the scanning software from the manufacturer to “descreen” the image and this basically eliminates the interference:
NoVsDescreen

Great!  Now I have pretty acceptable looking scans.  At least I have the basics of what I expect.  I’d taken some other stats before on scan time and file size if images are saved as JPG.  Here they are:

Test Kodak Canon
600dpi scan speed 35s / page 1:07 / page
300dpi scan speed 15.5s / page 18s / page
200dpi scan speed 6s / page 18s / page (again)
150dpi scan speed 5s / page 10.5s / page
600dpi file size not measured 5MB
300dpi file size not measured 1.2MB
200dpi file size not measured 600KB
150dpi file size not measured 300KB

Well that’s discouraging, but maybe not surprising. The Kodak is *dramatically* faster.  Making matters worse – the above measures are for the Canon scanner when the moire interference pattern is *not* suppressed.  With the interference pattern suppressed (which is really the only acceptable way to do this), the scan speed is >1 minute per page every time.

Finally, I wanted to decide on a scan DPI.  With the moire suppression enabled it doesn’t seem like I’m going to sacrifice any time on the project if I choose to go with a lower DPI, so all I need to do is figure out what would be acceptable. For archival purposes it seems like the only reasonable thing would be to go as high as possible but something tells me 600DPI (or higher, I think that I could do 1200) is just not really going to benefit anyone ever and it is would almost definitely make this take up even more of my time (in terms of initial processing and any post-processing) so I am planning on 300DPI or lower.  To make the call on this, I noticed that the Canon software is capable of taking some input files and generating a searchable PDF. I can’t stand PDF as a format but there’s no denying that this would be cool and handy, so I don’t want to choose a scan option with low fidelity if it seems that I might one day sacrifice that ability.  A couple tests on this and it turns out that the generated PDFs I make of 200DPI input files sometimes cannot find input search strings that I enter for people’s names in race results that are very clearly words on the printed page but at 300DPI in a handful of tests I didn’t find any misses.  Therefore: 300DPI it is.

To summarize:

  1. Canon wins vs. my Kodak. Other scanners will probably yield different results.
  2. It is absolutely necessary to turn on the descreen operation to reduce moire interference (and this is only available in the printer’s driver / software, not as a generic TWAIN device, it seems)
  3. Super-high DPI isn’t worth my time. In fact, did I mention how stupid it is that I’m doing this?
  4. But 300DPI seems to be the minimum to be able to make text-searchable PDF files and have that work.

I have a few more things to research before I get going, but I’m well on my way with these findings!

Comments

Inventory of Northwest Runner

IMAG0190

UPDATE: 6/4 notes inline with back issue notes thanks to Glenn Tachiyama. If I can pool Glenn’s issues with the issues I already have, this would be a complete collection from 1983-present.

To start out the project, I took an inventory of the issues I gathered from Martin. I’m still collecting details on what formats the various back issues are in, but at a high level:

  • the oldest issues are only available in the print copies (which I have)
  • newer issues (from something like 2000ish on) are available in some digital format

So the right thing to do is to scan the oldest issues and try to work with the digital format of the new issues and make directly consumable digital copies of those.

I don’t have access to the earliest volumes.  These date to the early 1970’s and if anyone has access to these, I would be very happy to digitize them, but I’m going to assume they are lost forever.  Here’s a stock of what I HAVE or what is MISSING:

  • Volumes 1-3: all missing
  • Volume 4: HAVE issues 5, 6, 7, and 10
  • Volume 5: HAVE issues 2, 5
  • Volume 6: MISSING 3, 5, 7, 11
  • Volume 7: MISSING 2
  • …note – all future volumes / years indicate issues that are MISSING…
  • Volume 8: 7, 10+
  • Volume 9 / 1981 (volume numbering changed this year): 1, 2, 4, 8, 9, December
  • 1982: May –> Glenn is also missing May, issue never printed?
  • 1983-1984: complete 🙂
  • 1985: July –> Glenn has this 🙂
  • 1986: complete 🙂
  • 1987: January –> Glenn has this 🙂
  • 1988: February –> Glenn has this 🙂
  • 1989-1993: complete 🙂
  • 1994: June, November –> Glenn has this 🙂
  • 1995-1996: complete 🙂
  • 1997: February, June –> Glenn has this 🙂
  • 1998-1999: complete 🙂
  • 2000: December –> Glenn has this 🙂
  • 2001: complete 🙂
  • 2002: September, October –> Glenn has this 🙂
  • 2003-2005: complete 🙂
  • 2006: February
  • 2007 on: assume digital copies exist

Comments

Archiving Northwest Runner

A couple weeks ago I got this idea that seemed great at the time. “Northwest Runner is a really valuable resource for runners in Seattle and I am positive that there is a ton of great history in there that should be preserved and made more publicly available. I should scan all the back issues.

It’s that last part where this may have taken a turn for the worse.  Anyway, I got in touch with long-time editor and publisher, Martin Rudow, and today I picked up a trunk full of back issues. I’m going to take some notes on this process and archive them in my blog for posterity and as I come up with questions that might be interesting for runners or hobby archivists.  This will probably start with a background of what data is available, go into technical questions / notes / challenges / discoveries, and hopefully just be kind of interesting.

I’ll try to remember to tag all the posts as “nwrunner” so that interested readers don’t have to wade through my extensive and deeply crazed rants on the current state of technology.  Wait…that’s the unabomber…not me…

Comments (2)

« Previous entries Next Page » Next Page »