Andrew Que Sites list Photos
Projects Contact
Main
   I was having some problems using Python 3 with os.O_DIRECT when opening a file. 
    inputFile = os.open( fileName, os.O_DIRECT | os.O_RDONLY )
    try :
      with mmap.mmap( inputFile, 0, access=mmap.ACCESS_READ ) as inputMap :
        hasher.update( inputMap )
    except ValueError :
      # Value error means the file is empty.
      pass

   Problem was, I kept getting a Permission denied error about not being able to open the file.  Turns out that there can be alignment issues when using the O_DIRECT flag.  The example for memory mapping I followed used this flag, but it isn't needed.  Everything worked fine once it was removed.
   Our water heater has been dying for sometime and after a lot of searching I finally found someone willing to install the tankless replacement I wanted.  I worked form home to be here for the plumber.  Took him a little over 4 hours to install the tankless heater and water softener, but it is done. 
   The selection of the tankless heater was done for a number of reasons.  I had a 100,000 BTU unit at the Garage, selected because it was small.  That worked pretty well although the water flow was low.  A lot has changed since 2003.  Modern units are condensing meaning higher efficiency and more power.  Most units now are are just under 200,000 BTU.  The one I selected was highly reviewed.  I wanted whatever water heater we used to be condensing as the old water heater was the last item to use the chimney, and I don't really want a chimney forever.  We also have 6 people living here, and hot water has been an issue when people shower back to back.  This heater should be able to address these issues.
   So far I'm pleased with the performance.  The standard 120°F/49°C water temperature is not quite as hot as I would like but I can change that.  The flow rate seems fine.  It is more than enough for showers, although the bathtub fills much slower.  The water softener is only on hot water, but should prevent the lime buildup I'm sure contributed to the breakdown of the last heater.

May 08, 2019

Snapshot program

I have been using a program called backintime for making periodic snapshots of my source code. It’s really great for regular snapshots without taking up too much space. The basic operation is simple. An initial copy is made. Then the next time a snapshot is taken, it is compared against the previous. Files that haven’t changed simply have symbolic links made to the original file. Files that have changed get a new copy. This keeps the size of each snapshot quite small. The program also has a rotation fetcher that can remove snapshots after some period of time has passed. This is exactly the functionality I was looking for to make hourly snapshots of my source code files. If I messed up, I could simply restore from any previous time. Worked so well I decided to try it on my website. That’s where I ran into a couple problems.

One of the problems is that my backup machine will sometimes loss the connection to the web server. When it tries to do a backup, everything looks like it has been removed. So, it snapshots an empty directory. Then when the connection comes back, the snapshot looks like an entirely new system. It didn’t know the network drive wasn’t available, nor did it know it already had copies of everything.

I decided to look into what it would take to make a similar system that addressed these short comings. In my system, original files are stored by hash. Snapshots are simply symbolic links to these hash files. What this does is only preserve unique files. Files with duplicate content link to the same source. If a file is removed the link simply doesn’t get made in the snapshot. If the file re-appears, the link again uses the original hash file. And if the data is moved, only the links are updated.

I put together a quick Python script to try this concept out. Only took an hour or two before the basic concept was functional. I used a non-cryptographic hash function called xxHash. This produces a 64-bit unique hash for file content and is very fast.

May 05, 2019

Mayo de Mustache 2019—Day 3

   Another beautiful day for the May Mustache celebration.  We had several more bands play, including an act on tour from Pennsylvanian.  Around 4:00 we saw that rains were coming and started to move items inside.  Our timing was good and it had begin to sprinkle as we moved the last of the items.  Soon afterward it began to downpour.  We had two more bands, but the garage was still dry so it all worked out.
   It has been a lot of work for a long weekend, but a lot of fun.

May 03, 2019

Mayo de Mustache 2019—Day 1

   Today is the first day of Mayo de Mustache (formally Cinco de Mustache).  The weeks prior to this have been full of getting the house ready for the largest gathering of the year.  Today started around 11:30 am with a cleaning of the garage.  After getting it swept and the stage down, I let Xiphos take over while focusing on the yard.  I needed to get the yard ready for camping and for that I used the lawn mower to mulch fallen leaves.  Tents went up shortly afterward and before long the first of 18 bands of the weekend were playing.
   This is my leveling scope setup the other day when we moved the wood pile.  I learned how to use one back in 2002 and I loved the concept.  Picked this one up in 2008 when the flooding began in Burwood Park so I could check how much water rise was needed before the water threatened the Garage.  I don't get to use it much, but it is a great device for those times when they are needed.
   I usually don't share images from work, but this was the result of a coworker I found impressive.  It is a custom control board made for a prototype device.  Much cleaner than any prototype I've worked with before.  The system uses a BeagleBone Black single-board computer.  They are similar to a Raspberry Pi but have more I/O and analog capability.  The custom board added to it does level shifting for analog signals as well as having a power supply for one of the sensors for the device.  I enjoy working with people who love what they do.