3G (1) 8600GT (1) AI (4) API (1) apple (1) apple mail (1) atlassian (1) bambo (1) Bamboo (1) bloat (1) boost (1) bugbear (1) C++ (5) calling conventions (1) cdecl (1) CI (1) compiler (1) continuous integration (1) coursera (1) custom domain (1) debugging (1) deltanine (1) diagnosis (1) diy (5) DLL (1) dns (1) education (1) electronics (1) fail (2) fink (1) Google App Engine (3) hackintosh (1) Haskell (3) homebrew (2) icloud (1) ipad2 (1) jobhunting (1) libjpeg (1) linux (1) mac (2) mbcs (1) mechanic (1) memory (1) MFC (3) Microsoft (1) migration (1) ML (1) mobile (1) movi (1) MSBuild (1) music (1) naked domain (1) NLP (2) o2 sensor (1) obd (1) Optiplex960 (1) outlook express (1) photos (1) PIL (1) Project Euler (1) python (2) raspberrypi (3) soundcloud (1) stdcall (1) subaru (2) supermemo (1) supermemo anki java (1) sync (1) Telstra (1) tests (1) thunderbird (1) udacity (1) unicode (1) Uniform Cost Search (1) university (1) upgrade (1) vodafail (1) vodafone (1) VS2010 (1) vs2013 (1) VS6.0 (1) weather (1) Win32 (1)

Thursday, 6 August 2015

Apple iCloud killed my trusty MacBook

Well, after a three and a half month valiant struggle to upload my 12000 photos and 1000 videos- (180 GB of data in total) to iCloud, my trusty 2008 MacBook Pro has finally died, succumbing to the video chip glitch that a lot of machines of this generation suffered from. Apparently this was due to the switch to lead free solder in the manufacturing process, and can often be fixed by reflowing the solder by heating the logic board in the oven at 200 degrees C for 10 minutes. But that will have to be for another blog post...

It nearly got there, it managed to upload about 100 GB of data, until it got stuck.... according to my calculations 180 GB at 1024 MB/sec should have only taken about 17 days to upload 24/7 - not over 3 1/2 months!

The main problem is that instead of uploading everything to the cloud, and doing the video and photo thumbnail generation in the cloud, apple uses YOUR CPU time to generate 2 versions of every photo and video in your collection at lower resolution so they can be viewed on other devices, which pegs your CPU at 100% and puts a lot of thermal strain on your computer, not to mention making it impossible to use for anything else.

When it actually gets around to uploading  the files to the cloud, it maxes out your upload which kills your ADSL download speed. It was so bad that 2 weeks in to this sorry saga I decided to upgrade to cable internet.

Then, if it encounters a corrupt photo and video in your collection, it will just stall without telling you it is stuck. But you would never know, because Apple don't give you any useful progress indicators as to how the iCloud sync is progressing- the only way really to tell is that the CPU and bandwidth usage isn't getting red-lined.

I  managed to figure this out by using the Activity monitor to see what files the cloudd process had open, and noticed that it kept looping over the same set of files. When I tried to open each of these files using finder, I found one photo that crashed preview every time. I deleted this file along with all the version on other devices and my upload progressed from being stuck at 94 GB until it got stuck again at 100 GB.

You can see all the temporary files that are used by cloudd when syncing by opening the Photos library file in the finder with "Show package contents". When icloud photo sync is enabled, there will be a subdirectory in there called private,  which contains some sqlite databases that are used to track the file sync and a whole lot of directories named AAA, AAB, AAC, AAD, etc which contain the files that cloudd seems to be uploading in batches to the cloud.

There are also 2 directories for the lower resolution videos and photos that are used by the VideoConversionService and PhotoConversionService whilst they abuse your CPU and turn your laptop into a nice little room heater. Again, you can identify which files are being converted by viewing the VideoConversionService and PhotoConversionService in the activity monitor and going to the open files and ports section.

Another top tip, which got my upload briefly unstuck for a while it to repair your photo library by launching Photos while holding down the alt option keys.

I thought I would be able to consolidate all my photos in the cloud from all my devices. I think I ended up deleting the same photo from my library about 20 times trying to clean up my camera roll, but whether I deleted it from my phone, my computer or the icloud web interface it would never get deleted from my other devices.

basically iCloud photo sync DOES NOT WORK. I'm convinced it is all a big scam to sell iCloud storage plans, that they will never have to worry about actually being used.

basically, iCloud photo sync sucks.

Friday, 19 June 2015

Atlassion Bamboo Deployment plans are half baked...

I recently noticed that Bamboo now has deployment plans for deploying build artefacts. I had already  implemented deployment as another stage in my build plan, so I thought I would try my hand at making a deployment plan.

My deployment plan is a very simple windows script that remotely installs an msi on a target machine.

But guess what?

Deployment plans (unlike build plans) have no means to specify any necessary requirements of the bamboo agents that will run the plan. In this case, it is a windows script and NEEDS TO RUN ON A WINDOWS BAMBOO AGENT!

So my deployment plan runs on any available agent, including linux agents, where it fails completely.

Deployment plans are useless if you have both windows and linux build agents. You can dedicate an agent to deployment, but then you can't use it to build anymore.

Massive fail, Atlassian!

I found a workaround: include a MSBUILD job in your deployment plan, which will force it to run only on windows machines. For the solution file, choose any random name and pass /help as argument to msbuild, which prevents it from trying to open the bogus solution file.

Thursday, 12 February 2015

My new raspberry pi 2 model b arrived!

I really should write up what I did with my other raspberry pi in my other post, like I promised...

Basically, I interfaced it to my security intercom in my apartment via a relay so I could unlock the security door with my iphone, because I lost my security key and my strata wanted to charge me $150 for a replacement.

It has since been decomissioned as I have moved house(which meant I had to buy a new key from the strata anyway... but since the strata management had changed it was only $75), but I will write it up when I get around to it (could be a while as I have noticed that I haven't posted on this blog for a looong time!)

It is currently being repurposed as a time machine for backing up my macs. Not sure what I will do with this baby - its got 1gb of ram and a quad corm arm processor. It would be nice to hook up to my stereo to stream music via airplay or squeezebox... it probably overkill for that purpose!

Thursday, 24 October 2013

Visual Studio 2013 and the arrogance of Microsoft

I have just installed the recently released Microsoft Visual Studio 2013 and the most striking thing about it seems to be the sheer arrogance of Microsoft and contempt for its customers.

It needs Windows 7 or greater to install which is fair enough - even  though my corporate desktop is still Windows XP (not for long as it goes out of support in April 2014).

No, the ridiculous thing is that VS 2013 requires you to install IE 10 before it can be installed. That's right - you have to upgrade your browser to IE10 before you can install the IDE!

This is a complete dealbreaker for many corporate environments.

Fortunately here's a workaround windows command script. VS2013 seems to work just fine with ie8 so it seems to be a marketing driven decision only.


REG ADD "HKLM\SOFTWARE\Wow6432Node\Microsoft\Internet Explorer" /v Version /t REG_SZ /d "9.10.9200.16384" /f 
REG ADD "HKLM\SOFTWARE\Wow6432Node\Microsoft\Internet Explorer" /v svcVersion /t REG_SZ /d "10.0.9200.16384" /f 
REG ADD "HKLM\SOFTWARE\Microsoft\Internet Explorer" /v Version /t REG_SZ /d "9.10.9200.16384" /f 
REG ADD "HKLM\SOFTWARE\Microsoft\Internet Explorer" /v svcVersion /t REG_SZ /d "10.0.9200.16384" /f 

REG DELETE "HKLM\SOFTWARE\Wow6432Node\Microsoft\Internet Explorer" /v svcVersion 
REG DELETE "HKLM\SOFTWARE\Microsoft\Internet Explorer" /v svcVersion 
REG ADD "HKLM\SOFTWARE\Wow6432Node\Microsoft\Internet Explorer" /v Version /t REG_SZ /d "8.0.7601.17514" /f 
REG ADD "HKLM\SOFTWARE\Microsoft\Internet Explorer" /v Version /t REG_SZ /d "8.0.7601.17514" /f 


 The other act of corporate arrogance is that they have deprecated non-unicode MFC applications, with the lame excuse that the 64 Mb MFC MBCS libraries would bloat their 5.7 Gb distribution of VS2013. You can still build MBCS applications but the libraries need to be downloaded separately from here.

Porting a legacy app from mbcs to Unicode is a non-trivial task... especially if there is direct pointer manipulation that assumes 8 bit chars sprinkled through the code base. This would be a whole lot of pain for zero gain.

Friday, 18 October 2013

More than I ever wanted to know about Win32 calling conventions...

Just spent the last day trying to debug a weird bug - one of those works in Debug builds but crashes in Release builds issues in a C++ MFC application. After tweaking some compiler settings I discovered it wouldn't crash if I disabled optimizations in release mode, but rather than doing that I thought I would dig down to find the root cause.
After much setting of breakpoints and attaching the debugger I found it was crashing in a call to CString::Format() - but only subsequent to a call to a function in 3rd party DLL that was used for encryption. The origins of this DLL were long ago lost in the mists of time so unfortunately there was no documentation.
The function was called by dynamically loading the DLL and calling ::GetProcAddress() to obtain the function pointer, which had the following signature:

typedef long ( __stdcall* LPFNDLLENCRYPT)(const char *, const char *, char *);

is used to call most of the Win32 apis. This calling convention can generate smaller code, as the called function performs the stack cleanup code.

__cdecl calling convention relies on the caller to cleanup the stack frame, and since functions are typically called from more than one place in the code, the additional stack cleanup code bloats the code.

On a hunch, I changed the function signature to:

typedef long ( __cdecl* LPFNDLLENCRYPT)(const char *, const char *, char *);

and lo and behold - it fixed the bug!

It seems the DLL was using __cdecl calling convention and expected the caller to cleanup the stack frame, but since the function pointer incorrectly declared it as __stdcall the compiler was not generating the stack cleanup code. This left the stack in a corrupted state which caused the next call to the CString::Format function in the MFC DLL to fail. Presumably the Debug build was more resilient with respect to stack corruption.

Here's a really useful article on calling conventions on CodeProject which explains calling conventions in more detail than you would ever want to know.... unless your debugging weird bugs due to calling conventions!

I love C++!

Tuesday, 17 September 2013

MS Compiler Bloat and broken MSBuild 4.0

Windows XP goes out of support next year so my work is finally upgrading to windows 7. This has provided the impetus to port our VS6.0 project to VS2010 as VS6.0 will not install on Windows 7. The port was fairly uneventful, just a few minor fixes because C++ wasn't standardized back in 98 when VS6.0 was released, so the code would not compile without a few tweaks.

The project is fairly large and takes 30 mins to build via Atlassian Bamboo build server. Imagine my dismay when I found the VS2010 version of the project took 90 mins!!! 15 years of compiler progress and they have managed to slow it down to one third of the speed!

So, I thought.... MSBuild has support for parallel builds - lets give that a crack! Unfortunately, it failed to build. After wasting a day I found this.
Apparently MSBuild 4.0 has a bug where it doesn't respect the project dependencies, so it gets the build order wrong.
Anyway, if you are using Bamboo with VS2010, use the DEVENV.EXE task for building projects, not MSBuild - it's broken!

Tuesday, 27 August 2013

bye bye vodafail

This month I inadvertently went over my data quota on my Vodafone service and copped a bill for $210. The culprit was my 3 yo son watching Thomas videos on YouTube.

I bought a data booster pack -2Gb for $30 dollars - that's $15 per gig, but it was too late, they had already charged me at a rate of $250 per Gb!

Since I am no longer under contract I'm free to leave for a player like Boost, which is offering unlimited calls and SMS with 3Gb of data - which is twice the data allowance of my Vodafone plan for only $5 per month more, and no chance of running up an excess data bill. I called customer support, thinking they would try their best to keep a happy customer, since they have been bleeding customers since the vodafail network debacle , but no... all they did was try to upsell me on plans. I even told them I was thinking of porting out...

Anyway, the port from Vodafone to Boost took less than an hour. Bye bye vodafail!