Migrating shravanj.com to DigitalOcean

For the past 6 years I have been hosting shravanj.com on a Raspberry Pi through a residential internet line. I first started out with a first generation model B, then upgraded to a 3rd gen model B. There was a massive speed upgrade between the two but this was later offset when I switched from a symmetrical fiber line with 75 Mbps up/down to a copper cable line with 400 Mbps down and 20 Mbps up. This, tied to the limited resource of the Pi, have become performance bottlenecks to my site over time. Over the past few days I have been looking around at moving my site to an off-premises host when I came across DigitalOcean’s incredibly well priced $5/month droplet which looked perfect for my needs. Today I migrated my entire site in about 45 minutes with just a few steps:

1. Create a new droplet with a Linux VM
2. Copy web data over SFTP
3. Copy and apply Apache vhost configs
4. Configure iptables for firewall and setup fail2ban
5. Point DNS to the droplet’s public IP

I am shocked at how easy it is to migrate static websites over to DigitalOcean, and the performance improvement is staggering. Prior to the migration, my homepage would take around 5 to 10 seconds to load; I am now seeing page loads in under a second. This is incredible for just $5 per month! Overall I am very satisfied with the platform and value, and will likely be using DigitalOcean for the foreseeable future.

Advertisement

Happy New Year!

We’ve just come to an end of a decade, it’s amazing how time just flies by. Here’s to great new year and an even greater decade.

Enabling CORS Support in .NET Core 3.0 Web API

CORS or Cross-Origin Resource Sharing is a web technology that allows cross-origin (read: requests coming from a different domain) API calls and resources to be shared. Typically, browsers protect against cross-origin calls but with CORS enabled, your browser will allow these requests when certain header values are returned by the request. This is especially useful when making AJAX calls to another domain’s API. When you are consuming someone else’s API, this technology has likely already been enabled on their end, but what about if you are writing your own Web API? Fortunately, development frameworks such as .NET Core have CORS support builtin via middleware. Microsoft provides some solid documentation on it here:https://docs.microsoft.com/en-us/aspnet/core/security/cors?view=aspnetcore-2.2, however, this only works up to .NET Core 2.2. If you are working with .NET Core 3.0 like I am, you will notice that the listed instructions for enabling CORS does not work. This actually has to do with the order of the setup calls being made, and which methods they are made in as shown in this issue on GitHub: https://github.com/aspnet/AspNetCore/issues/16672

The correct way to enable CORS in .NET Core 3.0 is as follows. Note that this configuration does the following:

• Allows any origin (making the API fully accessible to any website or program that wants to call it
• Allows any HTTP method whether it be GET, POST, PUT, OPTIONS, DELETE, etc
• Allows any headers like Content-Type, Authorization, etc

You can limit these by explicitly listing which methods, headers, or origins you want to use, take a look at the Microsoft documentation for the syntax (it’ll still work in Core 3.0 as long as you follow the correct order below).

        public void ConfigureServices(IServiceCollection services)
        {
            // Setup services (use AddCors after AddControllers and before AddMvc)
            services.AddControllers();
            services.AddCors();
        }

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }

            app.UseHttpsRedirection();

            app.UseRouting();
            
           // This call MUST be made between Routing and UseAuthorization
           // For your API to be completely accessible to any public consumer, you should allow requests from any origin
           // You can add restrictions for allowed methods and headers, but in this case we want to allow them all
            app.UseCors(
                options => options.SetIsOriginAllowed(x => _ = true).AllowAnyOrigin().AllowAnyMethod().AllowAnyHeader()
            );

            app.UseAuthorization();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers();
            });
        }

Xamarin Tips and Tricks: Uploading iOS App Archives using Xcode

Xcode 11 drops support for Application Loader which was the de facto standard for uploading apps whether they were developed natively or through a cross platform framework like Xamarin. Microsoft has since updated their documentation to show how to upload apps directly via VS for Mac: https://docs.microsoft.com/en-us/xamarin/ios/deploy-test/app-distribution/app-store-distribution/publishing-to-the-app-store?tabs=macos

While this method seems convenient, it involves some additional setup. Since VS for Mac is a 3rd party application that needs to access information related to your Apple ID, you’ll need to generate an app specific password for authentication. This sounds easy, but you should note this also requires 2FA enabled on that account. I am a strong proponent for 2FA, but this may not work out so easily for everyone such as when your development device uses a different Apple ID than your Apple Developer associated Apple ID.  Fortunately, there is a workaround using Xcode’s app distribution feature If you are logged into your development Apple ID in Xcode (Xcode > Preferences > Accounts), you can actually upload directly through the Organizer (Window > Organizer). As long as you have already generated the archive for publishing under VS for Mac, your archive will show up under the Xcode Organizer and you can sign and distribute the app via the usual channels (App Store, Ad Hoc, Enterprise).

Issues with push notification device tokens on iOS 13 (including a fix for Xamarin)

While I was trying to fix some issues with push notifications on a backend system I noticed that the device token being generated from my iPhone X running iOS 13.1 looked quite strange as it was returning something that looked like this:

{length = 32, bytes = 0x965b251c 6cb1926d e3cb366f dfb16ddd … 5f857679 376eab7c }

When I was expecting it to look more like this:

<965b251c 6cb1926d e3cb366f dfb16ddd e6b9086a 8a3cac9e 5f857679 376eab7C>

 

I soon found out, thanks to the articles listed below, that there is a breaking change in the way device tokens are accessed on iOS 13:

https://nshipster.com/apns-device-tokens/

https://onesignal.com/blog/ios-13-introduces-4-breaking-changes-to-notifications/

The fixes for Swift and Objective-C are provided, but what about for Xamarin? Thanks to Stack Overflow, we have the solution for that as well:

https://stackoverflow.com/questions/58027344/how-to-get-device-token-in-ios-13-with-xamarin

Note that this solution doesn’t take into account the formatting for iOS versions below 13, which is why I have written a code snippet that does:

	public override async void RegisteredForRemoteNotifications(UIApplication application, NSData deviceToken)
	{
	var token = "";
	/* Tokens generated on devices running iOS 13+ follow a different format which requires conversion for the backend to process them
	More info: https://nshipster.com/apns-device-tokens/
	*/
	if (UIDevice.CurrentDevice.CheckSystemVersion(13, 0))
	{
	byte[] result = new byte[deviceToken.Length];
	Marshal.Copy(deviceToken.Bytes, result, 0, (int)deviceToken.Length);
	token = BitConverter.ToString(result).Replace("–", "");
	}
	else
	{
	token = deviceToken.Description;
	token = DeviceToken.Trim('<').Trim('>');
	token = DeviceToken.Replace(" ", "");
	}
	if (!string.IsNullOrWhiteSpace(token))
	{
	System.Diagnostics.Debug.WriteLine("iOS Device Token: " + token);
	// Save the token using your prefered method (Xamarin.Essentials's Preferences works well)
	}
	}

view raw

DeviceToken.cs

hosted with ❤ by GitHub

 

There’s actually another issue that is causing problems with iOS 13 which has to do with the new apns-push-type. I will go over that in another write up, but for now I recommend reading the article from One Signal as it goes over this. As always, thanks for reading.

WWDC ’19 Round-up

Although I wasn’t able to make it to WWDC this year, here is a quick roundup of some of my favorite things announced:

iOS 13 and iPadOS

iOS 13 sees some evolutionary improvements and new features, big ones including Dark Mode and Sign In with Apple. iPadOS brings a revamped OS designed for the larger displays offered on across the iPad lineup. I honestly wasn’t expecting a new OS for the iPad but damn it look fantastic. Features like a full file manager and downloads in Safari brings the iPad one step closer to replacing laptops for many users.

 

macOS Catalina

In addition to the crossover of unified iOS apps into macOS and the ability to ship iPad apps to macOS, one killer feature that really wowed me was Voice Control. The demo video is simply mind blowing and really show how far Apple has taken user accessibility. This feature goes a step further and works across macOS and iOS. Take a look at this demo, just wow: https://www.youtube.com/watch?v=aqoXFCCTfm4

 

iOS Development Tools

SwiftUI looks like a game changer. A WYSIWYG interface builder with code generation, live previews, and declarative syntax? Yes please! https://developer.apple.com/tutorials/swiftui/tutorials

 

Mac Pro and Pro Display HDX

We go back to the cheese grater Mac Pro era with the latest evolution of the Mac Pro. Up to 28 cores, 1.4TB of RAM, 4TB of blazing fast flash storage, and a 1400W power supply. I need one of these, along with the $6000 6K Pro Display and it’s $999 stand. Yeah its pricey but oh man it has been such a long time since Apple has released such a great looking piece of pro hardware.

Lenovo ThinkPad E570 Review

I have been an avid MacBook Pro user for the past 5 and a half years. After purchasing my first MacBook Pro in 2013 I thought I would never buy another Windows laptop again after the nightmares I had with them. Yet here I am, reviewing a ThinkPad. I bought this solely as a work machine since I simply needed a dedicated portable Windows development environment. I do put my Dell desktop to good use but it isn’t portable and I travel quite often so it became more of a necessity than a want. After an hour or so of research I came across the ThinkPad E570. I have always held ThinkPads with high regard, with my first experience messing around with my dad’s IBM ThinkPad when I was a kid. The TrackPoint was the coolest little thing to me. Since then coworkers have held the latest T series to the highest regard, recommending them as the laptop to get sans a MacBook Pro if you need Windows. Considering I spent quite a bit on the 2018 MacBook Pro I figured I keep a tighter budget for my Windows laptop. The E570, and really the E series altogether was something I never heard of. I thought there was the T series and thats it. I figured a Core i5, 8GB RAM, and 256GB PCIe SSD was a decent deal for $630 shipped from Amazon. I soon learned that the E series was the “budget” lineup of ThinkPads and purists say it’s not even really a ThinkPad because the build quality isn’t even close to a T or X series. I figured for the price I can take a chance. Here is a breakdown of some pros and cons:

PROS

• Good specs for $630
• ThinkPad reliability (or at least perceived reliability)
• Fast Samsung PCIe NVME storage
• Decent 1080p 15.6″ display
• Fingerprint reader for Windows Hello
• Comes with Windows 10 Pro
• Good selection of ports

 

CONS

• Cheap plastics and fingerprint prone matte finish on lid
• Heavy battery that equates to just ok battery life with mixed use
• Lenovo junk preinstalled
• Fans come on quite often and gets pretty hot
• Wall charger cable is tiny and the connector could be better

 

Now it may seem that the quality is just as bad as everyone says but hear me out, I can’t really complain much for the price and competing laptops from Dell or HP will have worse build quality and/or specs for this price point. It certainly isn’t a T series and won’t hold a candle to any MacBook (perhaps every) in terms of build and finish but I don’t think it’s worth skipping this computer over. It seems simple to repair and could be durable enough for at least 2 to 3 years of moderate to heavy use. It’s kind of refreshing owning cheaper, plasticky laptop that you can throw around. I have only had this machine for a day so I can’t comment on long term use but I would say I already have a good impression. I would recommend this laptop to anyone with a budget of under $700 and wants the cheapest way to buy into the ThinkPad brand.

Tech update: 2 new laptops!

It’s been a while since I last posted as I have been super busy with classes and a new job I have taken up. The 2 new laptops are as you guessed related to the later. First off is a new MacBook that replaces the Late 2013 that I will likely always regard as my favorite computer I have ever owned. Many fond memories and important projects were made possible thanks to that fantastic machine and will now go on to serve someone else as a refurb. Replacing it is a beefed up 2018 MacBook Pro, configured as 15 inches with a 6-core Intel Core i7, 16GB RAM, 512GB PCIe SSD, and Radeon Pro 555X graphics. I won’t be doing a review since there are tons already out there but in short this thing is pretty damn powerful. It’s still thin and relatively light but my only complaint would probably be the keyboard, its just too loud for a laptop keyboard. The giant trackpad with Taptic Engine is responsive as ever albeit a little too big but not complaining and the lack of ports hasn’t really been much of a problem for me. No more MagSafe is a bummer but its nice to be able to charge on either side. Now, you are probably wondering what the other laptop is and its a ThinkPad! My new job requires a Windows machine and since I travel often I figured why not get a dedicated laptop instead of running a VM. Considering the new MacBook was a pretty penny I went for the more value minded ThinkPad E570 with a dual core i5, 8GB RAM, and 256GB PCIe SSD. I will be reviewing this in another post (only had it for one day). Initial impressions are pretty good and for $630 I’d be hard pressed to find a better built machine from Dell or HP. More to come.

The Hardware Level: Tales of hardware development from a software developer’s perspective

This past year I had the fantastic opportunity of getting to work with designing, developing, and integrating embedded hardware into a (soon to be viable) product. Prior to this I had absolutely no background knowledge in hardware engineering apart from the basic concepts of circuits from Physics in high school. I come from a software background so this was very new and exciting to me. I’m sure more experienced people will have much better advice but here are something I have learned from my stint at developing embedded hardware.

 

Open Source reference designs are your best friend when getting started

Without Arduino and Adafruit the project I am working on would honestly not be possible. The simple and approachable nature of many of their products is what helped me really hit the ground running. I designed our Bluetooth functionality around the Adafruit Feather nRF52 and other controllers around the Arduino MEGA2560 and UNO. The software development environment used to program these little things, the Arduino IDE, is so easy to use along with the C based programming language that saves a lot of time for implementing most functionality. Once we had working prototypes made in house via a slew of jumper cables, cut out project boxes, and lots of electrical tape and super glue, we proceeded to approach a PCB design firm to help us take this mess of wires into a single PCB. Although I was able to come up with the diagram and wiring I have no idea how to properly do a schematic capture or layout a PCB so I left that to the professionals. I did however learn a lot about PCBs themselves such as how they are made, what vias and traces are and how they connect things, different mounting technologies, and more which was super interesting.

 

Debugging hardware is nothing like debugging software

With software you can pinpoint control flow or data issues via a debugger or just print statements whereas with hardware everything is about the flow of electricity and signals, neither of those things can be debugged with software. If you are interested in getting into hardware you will need a multimeter for basic debugging. I recommend buying something decent since the quality of the tools you work with matters a lot more when working with hardware (more on this later). Fluke is the go to brand for quality multimeters although they are a bit pricey. There are some features that generally mark up the price of the multimeter regardless of brand, such as auto ranging which is not required but is a handy feature. Once you buy one, learn how to use it (they are pretty easy to use) and practice using it on simple circuits. Multimeters help you ensure power is going where it should with the correct amount of current at the right voltage, but what about making sure signals are ok? You can ensure that a signal is even being produced (since signals are just pulses of electricity) with a multimeter, but it’s hard to tell if you have the correct modulation and such. In these cases you will need a oscilloscope or a logic analyzer, depending on what you are trying to probe. For example, say you want to check a serial line to make sure data is being transmitted, the first step you’d do is make sure your RX and TX lines are alive with a multimeter. Once you know they have power you can use a logic analyzer or a scope to actually read the signal and even data when you use a logic analyzer. Overall, diagnosing hardware problems generally takes longer and is more involved compared to software since there are multiple things to probe with multiple pieces of testing equipment.

 

Read the datasheets for all the components you use, not just the ones that you suspect to be causing a problem

Once of the PCBs we had designed was based on the reference design of the Arduino MEGA 2560 revision 3. It’s basically the same circuitry with some our own added sensors built in in a smaller package since we don’t need jumper cable headers. You would think that since it has the same hardware it would be programmable just like the MEGA right? It’s a lot more complex than you would initially think because the MEGA comes with the microcontroller preconfigured and ready to be used with the Arduino IDE from the factory. Our PCB came with a blank Atmel ATmega 2560 as it shipped from Atmel with none of the configuration needed for it to talk to the Arduino IDE or even program via the ISP. After days of struggling, not being able to figure out why my AVRISP v2 programmer was not able to communicate with the mega 2560 I finally took a close look at the 400+ page documentation for the ATmega 2560. I found that from the factor, the mega 2560 is set to use an internal oscillator with a clock speed of 8 MHz. I needed the microcontroller to use the external 16 MHz crystal oscillator just like the reference Arduino design does. There are some fuse bits that you need to set to configure your clock signal source, but I need to get my programmer to talk the the mega 2560 before I can set the fuse bits. I then found out via manufacturer of the programmer (Pololu) that the frequency of the programmer needs to correspond with the current frequency of the microcontroller you are targeting. Since I tested my programming process out with reference Arduino hardware the programmer worked just fine because it was set to a high frequency that corresponded to a 16 MHz system clock, whereas my mega 2560 was running at 8 MHz. I lowered the frequency of the programmer and bingo I was able to communicate and set the fuse bits and got the external oscillator working. I was so sure that it was just the microcontroller the whole time I never would have guessed I need to make some changes to the way the programmer works. Moral of this story is check the documentation for all your parts, not just the one you think may be the cause of your problems.

 

When in doubt, consult the forums

There are so many great resources online for professional engineers and hobbyists in regards to hardware. If you have a problem, chances are someone else as had it too and there is a solution for it online. The Arduino forum, AVR freaks, and the hardware section of Stack Exchange have some great info. Try to search for broader solutions at first then narrow it down to the exact problem you are having.

 

I’ll try to add more points over time.

 

Helium + MEMS Oscillators = iOS device failure?

A system administrator noticed that all iOS devices in the medical facility he had been working at stopped working after a recent installation of an MRI machine. Turns out this was not related to any electromagnetic interference, but rather helium of all things. Below is one of the most interesting things I have ever read about electronics:

Original Post: https://www.reddit.com/r/sysadmin/comments/9mk2o7/mri_disabled_every_ios_device_in_facility/

Follow-up with some interesting comments from other people who have had strange bugs that were also hard to track down: https://www.reddit.com/r/sysadmin/comments/9si6r9/postmortem_mri_disables_every_ios_device_in/

The tl;dr is that there was a helium leak during the setup of the MRI. MRIs use very large, powerful magnets and liquid helium is used to cool them. The coolant was leaking which spread into the HVAC system in the facility. Many iOS devices use MEMS oscillators to generate a clock signal rather than using quartz. This is for packaging and cost saving reasons since they are smaller (thinner) and cheaper than a standard quartz oscillator. Quartz oscillators are hermetically sealed in metal to prevent interference but MEMS oscillators are usually sealed in plastic. Helium can permeate through plastic leading to the interference with clock signal, and since the clock signal is vital to numerous parts of an IC, the iOS devices stopped working. I do recommend reading the entire post though, it’s super interesting.