Past Entries...

100 posts since April, 2016!
posted this in: General, Servers, Software, Technology
863 Words

This is mostly a personal note on how to setup a workflow with my Web Development (PHP/MySQL & Docker based) projects. Usually, with pretty much any project, the workflow goes as follows:

file

Figure: My workflow prior to this article



To host my own repositories of codes per project, I actually use GOGS which is short for Go Git Service – it’s written of course, in google Go, and is essentially a self-hosted Github clone. It’s by one of the devs from the Gitlab team, and it’s far more lightweight and easier to use in a personal scope, than Gitlab (Gitlab is still wonderful, but I think it’s better suited to teams of 2 or more people).

Git hooks are amazing!

The above workflow diagram though, is missing one really critical stage – getting the code to production – a.k.a. deployment. Typically on any deployment, after the above workflow, I’d either remote in, or set up a cron job to pull from master. There’s some problems with this method of doing things:

  1. I’m doing a git pull which is based on merges, and can really cause some shit if there’s a conflict (there shouldn’t be, but just in case)
  2. I’m remoting into the server each time I have to pull for some commits, this takes up time. I’m a serial commit/push-as-save person
  3. It’s completely against the ethos of a developer. If I’m doing something repeatedly, I should find a way to automate it!

So, enter stage – Git hooks. Git hooks are amazing! They’re actually natively supported by Git (powered by Git) and I only really just started learning about using them. I vaguely recall encountering them earlier in my growth as a dev, but I must’ve shelved it at some point and given up trying to learn hooks (probably around the same time I cracked it at Drone CI, and Jenkins CI/CD).

Anyway, the overall concept of using a Git Hook is that I reach the final stage of the workflow I drew at the start of this article Git push to remote repository – the idea is that the Git repo then registers this push in something called a post-receive hook. It then performs some functions and essentialy plonks a pull of the latest code of my repo, into the production environment.

At first, I started off with something super simple, from my jtiong.com (this website!) repository as an example:

#!/bin/sh
ssh [email protected] 'cd /var/www/jtiong.com && git fetch && git reset --hard origin/master'

Unfortunately, this didn’t seem to work. I kept getting remote: Gogs internal error messages, and figured out it was something to do with my SSH keys not working in the authorized_keys and known_hosts files of the docker container to server shell and vice versa. After a lot of Google-fu and tinkering around, I eventually came up with the following which worked (note, it’s been edited to be a generic solution).

#!/bin/bash
ssh -o StrictHostKeyChecking=no [email protected] -p 22 -i /home/git/.ssh/id_rsa 'cd /project/folder/path && git fetch && git reset --hard origin/master'

It’s not entirely necessary, but I used the -p and -i options to specify both the SSH port and identity file used with the SSH connection (just for greater control, you should be able to remove them, your results may vary). The key section of the above command that I want to highlight is the -o StrictHostKeyChecking=no option that I’ve set. This got rid of any Host Key issues between the docker container and the host server for me. So if you’re encountering issues with your Host Key Verification or similar, this might fix your problems!

With the git command now, I used git fetch && git reset --hard origin/master instead of just doing a git pull. Why? Because git pull uses merge methods and can result in some conflicts with code, and issues that are just messy and a bad experience to untangle. Using git reset, moves the code’s pointer to a different commit without merging anything. It just overwrites it, making it slightly safer for deployment!

But of course… Why do things in just a simple way? This particular hook configuration is great for something like my personal site where I don’t mind if I’m pushing breaking bugs to production (within reason). However, when I’m doing work for clients, I need to be a little bit more careful – and I use a more typical productionstagingdevelopment branching method with my Gitflow.

Here’s what I use now:

And wonderfully – this lets me have separate branches, as required and updates the appropriate (sub)domain as needed! The wonder of Git Hooks has now streamlined how I develop projects, and deploy them in a much more pain-free manner! And so I dramatically take another step in my journey and growth as a developer haha 😛

posted this in: Gaming, Software, Technology
614 Words

eLeague.gg is rapidly growing behind the scenes, and this is especially true in regards to the tech development team behind the scenes.

Our current development team consists entirely of one person (yours truly) and my duties involve:

  • Coding behind all the site’s actions (more on this later)
  • Designing everything visitors see on the site
  • Writing all the content on the site
  • Designing the structure of the site, processes and user flow

Not to mention all the maintenance involved…! So the obvious solution, is to bring on board more people! The exceptionally talented1 :

  • PSmith a long time LAN party regular at my events and volunteer admin both at my events, and at PAX Australia
  • SirSquidness the well known server/network/LAN guy from Respawn LAN, Melbourne (also same for PAX Aus PC Area)

So…. What’s the problem?

Well, since I started out running LAN parties, and doing gaming events – all my sites have always been bespoke, from the ground up creations. All involving myself as the sole designer, and sole developer. I’ve never had to answer to anyone and in fact, never had to consider that someone else might come in and develop on my sites/projects. Again, this is certainly horrible practice and not something any developer should really do.

And so again, as eLeague.gg started up – guess what I did? Haha…

And now? What is the solution?

Git branches. I already develop everything and store it through a personal Git repository system (I use Phabricator for my Git repo management needs).

As it currently stands, I keep:

  • A branch called staging: this is for testing, development and experimental content creation.
  • A branch called production: this is for everything that is on the live eLeague.gg website.

While this stops me from accidentally implementing broken features (sort of) it’s still not ideal for three developers working all at once on eLeague.gg’s core code. So we’ve developed a way to organise ourselves in such a way that each developer is independent, but bases their code off a singular source of truth.

This looks like:

file

What’s more important is the flow of code as signified by the arrowheads in the diagram above. Developers should in theory, only ever be pulling and pushing features in both directions from the Dev(elopment) Branch. And all features should be internally vetted within the Tech Team before being pushed to the Pre(flight) Branch, and then onto the Production Branch if all goes well with the rest of the team.

Some caveats:

  • No more than a single developer should be working on a single feature at a time. If multiple developers are needed, then one dev needs to stop working on said feature, before another dev takes over.
  • There is a degree of potential data and code entropy over time between all three branches – dev, pre and produciton. As such, only JT (me) will have access to being able to merge ‘backwards’ from Production > Pre > Dev.
  • Setting up for three developers simultaneously also means we now have a bigger delay in rolling out features as everything needs to now be spec’d, documented and planned out.

I suppose it’s a great little situation to be in that I can start working on eLeague.gg in earnest with a team, and now I should hopefully be able to focus more on not just the technical development of eLeague.gg, but the rest of it too!


1: This list of people does not use names they uses their drivers’ licenses. Our team has many people with the same given names so to help differentiate, we refer to each other by our gamer nicknames.