Making change a habit is the best way to create an agile mindset

January 4, 2014 | 1 comment

“You must be the change you wish to see in the world.”
Mahatma Gandhi

Making change a habit is a powerful way of not just creating but maintaining an agile mindset. In the fast paced world that we occupy an agile mindset has come to be valued over many other attributes of successful companies. Agile is a word that is often used without a true understanding of the mindset that underpins its values.

Mindsets

A mindset is “a set of assumptions, methods, or notations held by one or more people or groups of people that is so established that it creates a powerful incentive within these people or groups to continue to adopt or accept prior behaviors, choices, or tools”. Another way of looking at that is a set of habits which an individual/group perform similar outcomes to. Agile has its origins in software development with the Agile Manifesto which was signed in 2001 and outlines the mindset needed to create successful software. The group that signed the Agile Manifesto stated that they value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

How your brain works

In order to better understand how you create the set of habits that form a mindset it helps to understand how the brain works. There are two distinct systems in your brain used for thinking as described by Daniel Kahneman in his book “Thinking Fast, Thinking Slow“.

System 1: Fast, automatic, frequent, emotional, stereotypic, subconscious
System 2: Slow, effortful, infrequent, logical, calculating, conscious

System 1 is where you spend most of your time and could be described as the autopilot part of your brain. Basically this allows you to do complex things without large amounts of energy and effort. System 2 is what you use when you encounter a problem that System 1 doesn’t recognise and this requires a different deeper type of thought.

Most people spend the majority of their time using the System 1 part of their brain. However like most things System 2 becomes more effective the more you utilise it. If you can take something from System 2 and engrain it in System 1 then you are then able to spend the energy you save on solving new problems. It also becomes the default behaviour for how you respond to this type of situation.

Habits

Another way to describe this type behaviour is a habit, defined as “A settled or regular tendency or practice, esp. one that is hard to give up” or simply something which has been engrained in System 1. They can be created and/or modified and a great description of this process can be found in “The Power of Habit” a book by Charles Duhig. He defines a habit of consisting of the following:

A trigger
A routine (what you do to respond to the trigger)
A reward

The easiest way to change a habit is to understand what the trigger and reward are and to replace the routine in the middle with a newer improved version. There are many triggers that exist in the software development lifecycle such as being stuck on a problem or lack of detail in requirements definition. When you start out most people have different routines for dealing with these type of problems. The ideal place to start to discuss these triggers is in a retrospective. In this environment it is great to understand the differing approaches and choose the most effective one for the group to adopt. The reward for this is the great feeling of shipping quality code on a regular basis.

Change a habit

I have found that to change a habit it is always best to start small and find a habit that will be reasonably straightforward to improve. This allows people to understand the process and see the benefits of the approach. I have also found that small regular conversations are much more effective for changing habits than large infrequent conversations. It takes a while for the habit to form and change and being subtly reminded on a regular basis is great way to reinforce the correct behaviour.

In order to create a mindset for a team you need to gradually change their individual habits to be consistent. This is not an easy or quick task but one that delivers an amazing amount of long term benefit to both the individuals and the company/team that they work for. This is probably the single largest factor in turning people into high performing teams due to the efficiency gains and consistency of outputs. If they have made a habit of hacking habits then they are constantly improving and helping each other to do the same.

Conclusion

It has taken me a few years to create a habit of change at Metro and these are some of the practical things that I did to help that journey. Initially I set a very clear high level goal of releasing software every day. This required a lot of changes and I wanted to ensure people knew what success looked like. I then setup regular retrospectives to create a change dialog. I then gave everyone my word that I would be a willing participant of this journey into change and set a clear commitment to remove impediments. This mainly involved getting the business and stakeholders to change alongside our journey. Once this was setup we started small, started changing and are iterating as there is no such thing as perfect process.

Music to write this code to

Fake Blood has some serious habits around create digging and turning old breaks into amazingly crafted mixes.

How to speed up WordPress Apache using Batcache Multisite and Memcache on CentOS

December 26, 2013 | 2 comments

I run a small WordPress network on an Amazon Micro Instance and this means that I need to stay on top of caching. Having had a look around at a lot of different options I decided that I would use Batcache and Memcache to provide me with in memory caching should one of my posts get popular. Batcache multisite capabilities means that I don’t have to stress about individual caching setups per site. Also Batcache is what WordPress.com uses to handle caching and you can utilise Memcache in a distributed way should you need to scale out from a single server.

Another reason I like it is that it is very non-invasive caching technique, I know that WP-Super-Cache and the way that it generates static files is probably a better fit for my blog as the load isn’t usually very concurrent. However that sounded quite straight forward and I like a command line challenge plus I have a few side projects that this might come in handy for. You also need to get this setup if you are ever deploying anything to WordPress.com which we utilise for our hosting at Metro.co.uk.

As I am running a network, I don’t want other users to have to worry about caching and its quirks. The nice thing about Batcache is that it doesn’t cache anything when you have cookies from the domain so anyone who is writing can see their changes immediately. I think this and its simplicity is probably the key advantage over some other methods of caching.

The other thing is that if you have root access to your server then it is reasonably straight forward and I thought that I would share how I did this here in case anyone else is interested.

First install all of the libraries that you need to run memcache and access it via PHP, it took me a few times to get this right so there might be a few libraries which are extra below.

sudo yum install memcached
sudo yum install php-pecl-memcache
sudo yum install zlib-devel
sudo yum install php-pear
sudo yum install php-devel
sudo yum install libmemcached-devel

This is the compiler I used for building

sudo yum install gcc

This will build the libraries that PHP needs to be able to access Memcache

sudo pecl install memcached

Now make sure that memcached will start on startup

sudo chkconfig memcached on

I also enabled memcache session handling but haven’t turned it on.

Enable memcache session handler support? [yes] : yes

I didn’t do as it suggested below as every time I did I got errors saying that it had already been registered.

You should add "extension=memcached.so" to php.ini

I had a look at the config and tried to lock it down to localhost only but it kept on throwing errors when I did so I left it, as all of my ports are locked down via the Amazon Firewall this shouldn’t be an issue.

sudo vim /etc/sysconfig/memcached

Now made sure that it would come back on startup.

sudo chkconfig memcached on

Now I followed these instructions for Batcache installation once I had downloaded it.

Upload advanced-cache.php to the /wp-content/ directory

Add this line the top of wp-config.php to activate Batcache:

define('WP_CACHE', true);

Tweak the options near the top of advanced-cache.php, I set mine to cache for 60 minutes after being hit twice in 10 minutes.

var $max_age =  3600; // Expire batcache items aged this many seconds (zero to disable batcache)
var $seconds =    600; // ...in this many seconds (zero to ignore this and use batcache immediately)

Optional Upload batcache.php to the /wp-content/plugins/ directory. As I ran Multisite I put this in the /wp-content/mu-plugins so it would be there for all of the sites in my network.

We have already installed the PECL memcached extension so you just need to add Ryan’s Memcached backend 2.0 to wp-content.

Test by reloading a page in your browser several times and then viewing the source once you have cleared any cookies from the domain you are hitting. Just above the closing tag in the head you should see some Batcache stats.

generated 524 seconds ago
generated in 0.374 seconds
served from batcache in 0.002 seconds
expires in 3076 seconds

Now feel free to tweak the settings around cache time to a period where you will reduce the load. Remember this is full page caching and you shouldn’t see it if you are logged in.

SEO Impact

The best part of installing Batcache has been the impact on SEO, below are my crawl stats from Google and you can see that the average time spent downloading a page flatlines around the beginning of December after installing Batcache. The really interesting thing is that the pages crawled per day went up at the same time. I think this is because Google now thinks that that server can handle a much greater load so sends it its bots on a more regular basis. Can’t say that this has had a massive impact on my SEO entries but it is a really good sign of the impact that you can have with a proper caching strategy.

Super fast and simple caching, if its good enough for WordPress.com its good enough for me. The below articles helped me along this journey.

Music to write this code to

Nothing beats repetitive beats to get down on the command line and Sasha playing at DC-10 for Circo Loco knocks that out of the park.

Heidi Rozen: It’s the business model, stupid

November 3, 2013 | Comment

Heidi Rozen gave a great talk at the DMGT Technology Summer School simply titled “Failure”. A Silicon Valley Alumni she has started companies, worked for Apple under Steve Jobs and now is involved in Venture Capital so she had her fair share of stories. I was really impressed with her perspective on development and seemed to have a very Lean outlook after all of her experiences. I collated the points of her talk below and most of it resonated very well with how I am currently thinking.

It’s the business model, stupid – first place to start when a company isn’t working
Most technical decisions are actually business decision
It’s not just about money, it’s also opportunity cost
Support the lunatic fringe
Understand the assumptions so you can quickly adjust when things don’t work out and they don’t
Innovate around what is core for you and outsource the rest
You are building for the platform of tomorrow, not of today
You must understand your customers ecosystem and fit in effortlessly
If you can’t win then recognise early and deal with it
Think backwards from your customers
Narrow the customer set to provide 100% solution
Understand the ecosystem and the business model
Build 95% solution and pay 5% is Google solution
60% of VC’s don’t return on their capital

Wendy Lea from Get Satisfaction on Customer Driven Development

November 3, 2013 | Comment

Wendy Lea the CEO of Get Satisfaction gave a presentation at the DMGT Technology Summer School on “The voice of the customer”. There were some great points on how they have infused passion for their customers into a customer driven development process. As their business is based completely on capturing customer feedback it was great to see how they have infused their company mission throughout everything that they do. I have collated the main points I took from the talk below.

“Listen -> Engage -> Iterate” is their mantra for development
“Advocate -> Discover -> Buy” is a common loop for new customers
You need to create and curate a community for product development (Facebook, Twitter handle)
The Cluetrain Manifesto – “The connectedness of the Web is transforming what’s inside and outside your business — your market and your employees.”
Create systems of engagement mashing in your external and external systems
Conversations created by real people, in their natural language are very powerful and discoverable
Bidirectional conversation can help that flow
Join the conversation:
- Embrace customer conversation
- Open these conversations
- Collaborate
- Apply actionable insights

Embracing the above thoughts into a customer driven development process is something that really got me thinking on how we might be able to apply this at Metro. Apart from the old letters page newspapers traditionally are more of a one way dialogue. I have been thinking of ways that we could apply some of the above and simple feedback forms are something we are going to begin trialling soon as we make further changes to Metro.co.uk.

Strategic Perspectives from Bill Raduchel

November 3, 2013 | Comment

I had the pleasure of spending some time at the DMGT Technology Summer School with Bill Raduchel. He had a long and interesting career before joining DMGT as a technology advisor to the board. I managed to capture some of his advice below and a couple of quotes which resonated well with me.

“Heroism is an act of stupidity that you happen to survive.” Bill Raduchel

Open source is the only way to get the best developers as they can learn portable skills
Proprietary systems are struggling
John Chambers CEO Cisco: We compete against market transitions not companies
You must get people loosely coupled but tightly aligned
You must have a vision and be able to sell it, you must communicate it in a way that all your employees can understand
Asking the right questions is never interference
You must set the boundaries and then get out of the way
Architecture is to empower your team, not constrain it
Commitment before success is the mentality that separates people in a startup from people in a corporation

“Mindset is a value that has become a habit.” – Bill Raduchel

How Metro built a metrics driven product development process

September 29, 2013 | Comment

I have spent the last three years at Metro constantly tweaking our development process to fit our environment and teams maturity. At the same time the product development process had remained reasonably unchanged. Someone comes up with an idea we all have a debate about how good/bad we think it will be and someones opinion wins out the day. However in the last nine months we have also begun to iterate on this part of our process to make it more of a metrics driven product development process. It has been a slower process than I had anticipated and thought I would share some of the learning around that here.

I think that the lack of established templates to follow such as SCRUM has made this a harder process as there isn’t a template to start from. I also think that in order to get it right the metrics need to be driven from the business, so you are dependent on them being engaged. The toolset that helps you move to this type of approach is also lacking and in the end we had to build/hack our own.

Our journey began by the business setting one key goal backed by a clear metric that the whole business could understand. This was easier said than done and were definitely parts of the business that feel left out by this singular approach. However just having one key metric in the beginning is very important as it aids focus. Metro’s was to have an average of 700,000 daily mobile visitors in September 2013.

The next phase was to have a look at all the data that makes up the key metric and to break it down further to show each area of the business what they could effect. Being as open and honest about where the numbers have come from at this stage is essential. There will definitely be questions about how the decision was made especially if it is a large goal and being open at this stage is the best way to combat that (even if it it is a vanity/stretch target).

As the largest part of the goal was going to come from metro.co.uk and that partly my teams responsibility we started by looking at our web analytics (Omniture) to see what data we were already collecting. We also looked to the content and search teams to see what metrics they were tracking as only by working together would we be successful. Common goals should bring people together to talk about frustrations/improvements, this alignment should provide major benefits. However it still surprises me in large businesses how little people actually talk.

I think part of this problem is the fact that so many parts of the business have their own language. Development as much as anyone contributes to this as it takes quite a lot of time to be able to converse with most developers. Marketing, Sales and Content all also have their own ways of working and language which can be just as daunting for developers. Getting the translation right between these different groups so they can talk enough of the same language to communicate effectively is a major business benefit. I think this is one of the reasons that startups are able to iterate faster as they have to talk the same language as you have a limited set of people doing a lot of of different things but all speaking the same language.

Having had and looked at all the existing metrics we decided that there wasn’t actually enough specific product based feedback mechanisms to measure the success of our work. So to understand the product more we had to build our own measurement/tracking framework on top of Omniture. This took some work to get right and then took even longer to get the tracking across our entire site. However once we were able to see the results of each of our improvements in both click perspective and overall impact on the wider numbers a really important feedback mechanism was created.

Measuring the impact was only the first part however we needed to get the data into a format that everyone could see and understand. This was to be a common theme throughout this process you go from having no data to having too much data that doesn’t make a whole lot of sense. Having dedicated resources and time to create dashboards that compiled this data into meaning was the next step on this journey. Then we created regular meetings that talked around this data and tried to put meaning around it. Product Performance was born and we scheduled it in for every two weeks. We get all of the product owners and stakeholders that are involved in product development into a room and talk through performance of the previous two weeks. We also do a product based standup twice a week where all stakeholders stand in front of the Kanban board and discuss what is being done and about to drop into the process. Lots of small regular conversations definitely provide more value and leverage for change than large infrequent meetings.

We also got large televisions that monitored real time traffic and put them up in visible locations so everyone could see what was going on and feel the excitement when a large story took off. Real time data is the next step in this process and the faster you can see the data the faster you can react to it. The next step after having everything measured was to begin A/B testing. This step again proved to be a much harder than we initially thought. As our site is news based and constantly changing we found we couldn’t really use a Javascript based framework like Optimisely as things changes too much. Plus the cost was very prohibitive for a site of our size. So we ended up building a simple framework that extended our measurement in Omniture.

The other key learning was as we release multiple changes everyday what do you test and what do you just measure. We got to a place that if you were working on an incremental improvement then A/B test otherwise just ensure that everything is measured. We were surprised as well that sometimes the A/B results were 50:50 but when we made then change anyway it had a longer term affect on user behaviour that was positive and not captured during the testing.

We also had some issues with our Google News Sitemap and indexation. This was out most experimented feature over months. We ended up creating a dedicated swim lane on our Kansan board just for this. We only ran one test at a time and then iterated to the next one. I think for harder more complex options running just one test at a time really helped focus in the results of each before moving on.

Running too many tests at the same time proved to be confusing and I ended up making one of the team the chief growth hacker and he became the go to person for what and how to run tests. Now that we are in a place with a nice testing framework and approach it was time to see if it would work towards a more major site redesign. We needed to break this big design down into testable chunks that would prove our hypothesis that people would interact more by scrolling than swiping. Creating stories that were just for learning felt like a natural evolution from a lean mindset.

Conclusion

It has been a really interesting nine months and we have managed to hit a goal that seemed completely unachievable only 12 months ago. I think we managed to create a measurement mindset where everyone asks what we are measuring before starting development. This added to the power of everyone pointing in the same direction, having a process that allows constant improvement and promotes communication has produced some amazing results. You should give it a go.

Start with one key metric
Break metric down to provide relevance
Measure everything
Create dashboards to distill data
Make the data visible to everyone
Regular meetings (short and often) to create common language
Try running some A/B tests
Keep changes small and regular
Communicate, communicate, communicate
Enjoy your successes

How to setup an AWS EC2 instance to reduce WordPress hacker attacks

September 16, 2013 | 1 comment

WordPress hacker attacks have been on the rise this year and although I haven’t been hacked (touch wood) they have managed to overload my site on a number of occasions. I have been on a quest to stop them getting in and also to reduce the instances of their repeated entry attempts taking it down. This has been a pretty long journey and one that I thought other people might benefit from so I have decided to share it here.

Monitoring

The first step that I took was setting up alerts in the Amazon EC2 console to send me an email if my CPU stays at 100% for over five minutes. You can do this by clicking on your server in the console and then hitting the “Monitoring” tab and then “Create Alarm”. This seemed to be about the right amount of time to stop too many false positives and usually indicated that something was up with the server.

The second set of monitoring I setup was New Relic, we use them extensively at work to monitor our servers and their free version gives you a load of really useful information about how both the server and PHP are performing. Think of it as a version of Google Real Time that allows you to look at the processes and speed of everything going on inside your server and application. They also have a really useful availability monitoring service built in which you can setup to ping your site every few minutes to see if it is accessible and a great iPhone App that sends push notifications when something goes down.

Logs

The next thing you need to do is SSH into your server and check your httpd access_log to see what is trying to access your server.

sudo tail -f /etc/httpd/logs/access_log

This will bring up what is currently happening on your server and you can see all of the requests that are coming in. If your site does go down this is the place to check right afterwards to see what the requests were that overloaded it.

Mod Security

I decided to install Mod Security along with the OWASP ruleset to give myself an added extra layer of protection from any issues that might get introduced into WordPress before a patch comes out. I have to say that this involved quite a steep learning curve and due to the amount of technical information that I write in my blog so it took me a long time to modify all of the rules to allow saving of posts amongst other things.

As I am running on the AWS Centos image and it already had the EPEL repository setup so all I needed to do was:

sudo yum install mod_security mod_security_crs
sudo service httpd restart

Once this has installed you should definitely start clicking around your blog and see what is broken, quite likely that actions like saving a post, updating a comments status and updating plugins will now throw 403 errors. In order to allow your site to function need to create a file called whitelist.conf in the following location.

/etc/httpd/modsecurity.d/activated_rules

You then need to populate it with rules that allow you to bypass the specific rules that are triggered when using WordPress. Mod_security isn’t aware of your logged in status and assumes every call is external so this throws up quite a few issues. I have included a link to the ruleset that I have built up over the last few months below. There are probably more entries than most people will need but due to the geeky and code based nature of my posts a lot of stuff gets blocked, I have also enabled XMLRPC so apps work as well.

whitelist.conf

Adding your own rules to whitelist.conf

Should you get a 403 error when performing any action on your site you should SSH to your server and via the terminal type in the below.

sudo tail -f /etc/httpd/logs/modsec_audit.log

Then when you retry the action you should see another entry in the log and you need to pick out the data that looks like this.

[id "960035"]

You then just need to add the following line to your whitelist.conf

  SecRuleRemoveById 960035

Under the correct Location e.g.

<LocationMatch "/xmlrpc.php">
  SecRuleRemoveById 981173
</LocationMatch>

This can be quite a laborious process and possibly the whole lot is overkill but have to say I have learnt quite a bit in the process. I also added the following to the bottom of my http.conf which bans an IP from attempting to login for five minutes after three failed attempts.

# Stop Brute Force Attack by banning IP http://www.frameloss.org/2011/07/29/stopping-brute-force-logins-against-wordpress/

        # This has to be global, cannot exist within a directory or location clause . . .
        SecAction phase:1,nolog,pass,initcol:ip=%{REMOTE_ADDR},initcol:user=%{REMOTE_ADDR},id:2323

                # Setup brute force detection. 

                # React if block flag has been set.
                SecRule user:bf_block "@gt 0" "deny,status:401,log,msg:'ip address blocked for 5 minutes, more than 3 login attempts in 3 minutes.',id:2323"

                # Setup Tracking.  On a successful login, a 302 redirect is performed, a 200 indicates login failed.
                SecRule RESPONSE_STATUS "^302" "phase:5,t:none,nolog,pass,setvar:ip.bf_counter=0,id:2324"
                SecRule RESPONSE_STATUS "^200" "phase:5,chain,t:none,nolog,pass,setvar:ip.bf_counter=+1,deprecatevar:ip.bf_counter=1/180,id:2325"
                SecRule ip:bf_counter "@gt 3" "t:none,setvar:user.bf_block=1,expirevar:user.bf_block=300,setvar:ip.bf_counter=0"

htaccess

The final part of this is the .htaccess changes that I made to stop people being able to post without a referrer as a lot of bots use this method as they are computer programs rather than real people. I also had two specific User Agents that had targeted me which I block all access to my wp-login page.

# Stop protected folders from being narked. Also helps with spammers
ErrorDocument 401 /401.html

# Stop spam attack logins and comments
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} .(wp-comments-post|wp-login)\.php*
RewriteCond %{HTTP_REFERER} !.*(blog.david-jensen.com|dev.mariamjensen.com|www.cafeontherye.co.uk).* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/5\.0\ \(Windows\ NT\ 6\.1;\ WOW64\;\ rv\:18\.0\)\ Gecko/20100101\ Firefox/18\.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/5\.0\ \(Parsley\ NT\ 1.0\;\ rv\:1.0\)\ Parsley/1.0.0.0 [OR]
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule .* - [F]
</ifModule>

Here are a few sites that helped the creation of this post.

Definitely interested in hearing if anyone else has any other good ideas around how to approach this issue or has any tips from me as I am certainly no security expert. However after the uptime for my server has definitely improved over the last while with all of this helping to reduce WordPress hacker attacks on this blog.

Music to write this Code to

Andy C delivers some seriously high paced hacker music to help you get through this code in a jiffy.

22 tips for managing successful Agile projects

August 18, 2013 | Comment

In the past year I have run two large successful agile projects that have delivered software using the Scrum methodology to build a minimum viable product (MVP) and then transitioned to Kanban once they have gone live. The first one was the Metro.co.uk responsive redesign and CMS migration which was delivered on time and under budget. Since going live it has been quickly iterated to deliver solid mobile and overall traffic growth for Metro. The second project is the Metro Play Casino which delivered its MVP as part of a large program of work that required us to get a fully regulated offshore gambling business started. This is now ahead of budgeted expectations and we are currently scoping the next phase of development. During both of these projects there have been some ups and downs but overall it has been a really positive experience. I thought I would share some tips that I have collated whilst managing these projects successfully using agile methodologies.

Initially just define a few clear high level goals to measure success that allow detail to be filled in at the appropriate time.
Start prototyping as early as possible, learning by doing is the most effective way to figure out what to do and also what not to do. Being able to actually use something as early as possible is very important. Even if this means releasing something different than your end goal.
Don’t break down every story in the beginning and add it up to find out how long the project is going to take, this will just leave you with an unmanageable backlog and most of the stories will be out of date by the time you come to develop them. We have found that the best approach is to get a group of experts in the room and play planning poker for the project as Dan North suggests calling it Blink Estimation.
Don’t be afraid to pivot your MVP or technology choices if the learning that you gather points in slightly different direction than you first thought. Knowing what you don’t want is as important as knowing what you want.
Clear and consistent goals facilitate quicker decision making. These enable decisions to be made based on logic and this decisioning can be distributed around the team and post validated. Any single point of decision making is going to really slow things down, especially important in the last couple of iterations.
If you are still developing a lot of new functionality in the last sprint be worried. Also leave a little bit at the end to tie it all together as things always pop up that you didn’t plan for.
A more traditional Project Management layer can really help with external vendors and internal communication. Keep project plans at the highest possible level for visibility and better decision making especially on dependencies.
Change your process, change it again. Retrospectives are really key to this process so don’t forget to have them after every sprint. Be agile about your process.
Quite often order comes from short periods of carefully orchestrated chaos, these can help teams form by giving people common frustrations. Listening to these and then adapting your process to remove them helps the team to feel empowered.
Get onto your production environment as soon as possible, no matter how similar other environments are they are never the same and issues will come out at this stage.
Get developers and the business talking directly if at all possible. Learning together through structured conversation delivers the best business value. These can be facilitated by a Business Analyst or Product Owner but they should not always be in the middle as the lack of knowledge transfer will harm the project. This is especially true when dealing with multiple stakeholders.
Bite off the biggest unknowns first and work on them to make them knowns. The best way to do this is to release something early and get real people to use your product. This doesn’t need to be MVP but a Minimum Learning Product that delivers some value to end users.
You can assemble a group of people but it takes time and effort to form a team. Setting achievable goals and then ensuring they are delivered with a sprinkling of pressure early is a great way to prepare for the final few sprints.
Restrictions bring out peoples creativity.
In my opinion design should be emergent as should architecture, initially go with the most established patterns however you need to give the team time to fix things if they turn out to no longer be the right choice. This allows you to focus on actual problems rather than solving ones you only think you have.
Automated testing should focus on smoke tests and high level indicators when things are rapidly changing.
Automate builds and deployments in phase 0 for maximum payback.
Constantly engage the business to make them define the problems they are trying to solve and then work together on the right solution for the project and the business in iterative steps.
If you are going to take any risks with the project take those decisions at the latest possible point.
Celebrate the team and their successes.
Things will change constantly so learn to embrace it and hopefully get to the place where you enjoy it. If you have problems detailed rather than solutions then these can then be adapted as you learn.
Simple things you haven’t done before are complicated to execute. Do not under estimate… or fall into the planning fallacy. Build complexity in over time.

Conclusion

Managing successful agile projects is something that can deliver serious business benefit if done with the right approach. The constant feedback that is available allows iterative learning and if your environment is setup to handle this will allow you to be very nimble. Quite often agile is an approach amongst a sea of different project management methodologies and I believe that if adapted properly to your environment can bring real benefits to most projects.

Music to write this Code to

Just Be (Bushwhaka!) delivers some serious house sounds in San Francisco making me wish it was sunny every time I listen.

Vrsus: “The future of real time voting” – DMGT Hackathon 2013

May 31, 2013 | Comment

About six months ago I was sitting in a CTO meeting across DMGT and they were trying to come up with ways that we could improve collaboration across the group. After a few suggestions around wiki based API pages I came up with the idea to run a Hackathon. In my experience the best way to get geeks to know each other is let them build stuff together. This idea stuck and luckily Oliver Parke had sometime on his hands and got the job of organising it all. We were very fortunate to be able to get the Campus London Workspace to hold the event which is a hot bed of startups and geekery. The event was run on the 23rd and 24th of May and included technical people from businesses across DMGT including:

Metro
Mail Online
Euromoney
dmg::media Sprint Suite / Dev Lab
Landmark
Evenbase
Broadbean
Wowcher

I was fortunate enough to also be able to participate in Haribo fuelled hacking and below is an overview of what we built and how we approached the whole event.

Vrsus: The future of real time voting

Trying to find a group in the initial hustle of the team forming stage was easier than I thought. My initial idea was something based around real time analytics but this quickly morphed into a real time voting tool to decide which the best cats on the internet are. The great thing about the real time approach was a chance for us to use a load of new technologies. So the dream team that we ended up with consisted of the following:

Mary Morgan (Euromoney Institutional Investor)
Frédéric Occédat (Mail Online)
Panagiotis Albanis (Metro)
George Old (dmg::media)
Me (Metro)

Once we had a general direction with regards to online voting we had an hour design session around our flip-board to try to conceptualise the flow through our app and what screens and actions would be necessary. This gave us a decent enough understanding of the requirements to put together a technology stack which consisted of the following tiers:

Amazon EC2 Ubuntu Instance
Twitter Bootstrap
Angular.js
Socket.io
Express
Node.js
Redis

As we wanted to be real-time and socket based most of the stack chose itself and we thought that Redis would provide a simple way of storing the data and retrieving the post via score. At this stage we started to get down to the detail of building the initial flows around being able to submit images for voting on.

I also started work on setting up the EC2 instance so we could easily pull from GitHub and keep the server running. There were a few things that I need to do in order to get the ports forwarded to HTTP and the Node process running in the background for which I used the following:

Forever – makes Node.js run as a background process
Rinetd – allows port forwarding on Linux so Node can run on port 80 and sockets on 8080

Next steps were to get all the pages setup and the logo done which took the rest of the evening.

Coming in the next day first thing we sorted was our Kanban board to get everyone working on the different bits that were required. This worked well through out the day as we finished the following screens.

I also managed to procure the domain name vrsus.me and then added the subdomain you.vrsus.me to it to give it a bit of entertainment. The second day disappeared far to fast and I focused on stripping down the homepage to be nice and simple and also on the create page the ability to preview the image as soon as it has been added using sockets. We decided to against our idea of being able to enter any URL and for us to go and scrape the Open Graph or Twitter tags from it to get the images from the page to just entering a URL of an image.

This proved to be a good idea as it took us the rest of the day just to get the basic flow working, due to Redis not having any relational abilities there it took a bit of time to figure out how to store both the complex hash around the upload as well as the score after voting. The ability for the list to change order in real time also came right down to the wire and although there isn’t a fancy animation the list view is socket based and will change order when votes come in which I think adds a lot to the application.

It was then time to give a presentation of how we had approached everything and I had managed to knock up the below in Prezi.

We won the ‘Most Technically Impressive’ category overall.

Future Ideas

If we were to have some more time to develop this application into something more fully fledged I think these are the area’s that we would focus on.

Ability to post any url and vote on it
Social integrations
Ability to put sponsored posts in it
General responsive and cross browser performance
Improvements to the animation
Tracking

I think that Vrsus could be utilised across DMGT in many different ways apart from just showing the best cat beards on the internet. Having a real time voting system that people could submit content to that allowed the best to rise to the top based on user interaction would be a valuable asset in my opinion.

If you are interested in having a look at the code you can see it on GitHub and the url you.vrsus.me is still live and will be so for the next month or so.

Video

DMGT Hackathon – Google Campus from Dan Buck Joyce – OFILMS on Vimeo.

Photos

Changing the Paradigm with Swipe

May 22, 2013 | 1 comment

I was very fortunate to attend the 2013 WordPress VIP Developer conference in Napa Valley this year. It was great to meet all of the people who use WordPress at scale across the media space and the lovely people behind it all at Automattic. Also crazy to see that the average development team size for someone on WordPress VIP is 2-3, considering many of these sites are serving millions of uniques a day that is incredibly lean.

I was asked to give a lightening talk (less than 5 minutes) on a subject and I decided I would give a quick overview of how we have approached building Swipe into metro.co.uk. This seemed to go down quite well though qz.com were on after me and what they have done with their front end is even more impressive. I thought it would be a nice idea to put my slide deck up somewhere for viewing.

UPDATE: Metro have decided to remove swipe from Metro.co.uk for these reasons.

One of the other talks the previous day was by Yuri Victor on why the Washington Post uses WordPress and if you haven’t already seem it then it is definitely worth the visit even for the typography alone.

Other great notes from the few days were to use the following for WordPress debugging.

Also use Grunt to automate the use of these two libraries for image optimisation.

Generating fonts to help reduce images/sprites can be a good idea or just use Genericons which have been done for you. Also I found out that script tags on a page are completely blocking.

Mindsets

How your brain works

Habits

Change a habit

Conclusion

Music to write this code to

SEO Impact

Music to write this code to

Conclusion

Further Reading

Music to write this code to

Monitoring

Logs

Mod Security

Adding your own rules to whitelist.conf

htaccess

Music to write this Code to

Conclusion

Music to write this Code to

Vrsus: The future of real time voting

Future Ideas

Video

Photos