The New Web Development

There are two extreme examples of what "a website" is today. The average website lies in the middle, but if you're building website's that look mainly like this:

You're likely to be doing the best thing for users and clients by building websites in a traditional manner:

• Have the server output one minified HTML page for the URL in question
• Serve one or two minified long-lived CSS files for the whole site
• Enhance the experience with some interactive shine via vanilla JS or a jQuery widget

The new web development stack (currently) will struggle to outperform what you're already doing for a text-heavy document centric website.

However, if your clients are looking toward having interactive web 'apps' more like this:

Then you need to jump on board the new development methodology, because traditional techniques will never provide experiences approaching this.

I don't build web-apps so I don't need to worry... right?

The web has always been dominated by static documents - and that aspect is always going to be a major part of the web. But less and less of that type of content will require the technical skill of a dedicated web developer to make. You'd better be offering some excellent bespoke design work going forward, that's for sure.

After you tap a link or type a URL the result you see on screen is moving toward a much more interactive "app" model, and away from the mostly static "document" model. Even "document" style sites are getting smatterings of app-like features as standard.

Don't get stuck in a couple of years finding you can only build websites that are nicely presented static documents - then look around and see that Software as a Service solutions are doing "well enough" for more and more of your former clients.

I'm a long-time web developer. Someone who's been crafting websites for almost two decades - and most of that professionally. I'm good at my job, and have been for a long while.

I know about document-centric best practices, and about web standards. About accessibility, CMS's, web servers, databases, browsers, networks, and operating systems. I know information architecture, design considerations, interaction principles, and how to solve problems for my clients and their audience. I can use Photoshop as well as anyone, and I can problem solve in various contexts. I have experience in a lot of things. Because you have to be all that to be a good web developer.

I'm also a web developer playing catch up - like many of my peers might be.

Not because I took my eye off the ball, but because a technology I have looked at over the last few years and rejected as immature or not my use case... has matured significantly, and is starting to become my use case.

I've kept up with a lot of technological change over my career, but it's my opinion that the biggest one to hit the industry since Responsive Design is playing out now. And it'll have as big an impact on the web as a whole as the Web Standards revolution that came before it.

I am not joking when I say this is a paradigm shift. The techniques, tools, best practices, and assumptions when making an app-style website vs a traditional document-style website are different. Dauntingly so. Because the modern web has vastly expanded scope compared to how things used to be.

The modern web developer is more a software engineer than what I would call a web developer. The app-ification of the web isn't a case of writing some HTML, authoring a CSS file, and using a new jQuery-like library to get that sweet interaction.

You are going to need to know how to program; like a real programmer. You are going to need to know some very different approaches to how you build, deploy, and maintain things. You're going to need to question the value of some of your strongest held best practices.

And you're going to need to get to grips with tooling. So much tooling. Because the new web development has matured enough to be dangerous to ignore, but not enough to be simple and entirely stable.

Building things for the new way is not much like the way you build for the old way. You will not be hand-authoring one or two files in a simple text editor, uploading them to a server, and wash your hands.

It melted my mind over the holidays while I got to the point of understanding what all these new things are supposed to do, and how they are supposed to fit together, and the high level concepts behind them. The world of setting up a development environment stack, and "build steps" and their labyrinthine configuration and dependancies. There are whole concepts that are new to me... all to get some pixels to show on a web page; which theoretically is my expertise.

Modern web development is the story of JavaScript going from an unstable ugly-duckling bit of browser based weirdness to what is now the dominant force in all of everything. From the tools you need to use to write things for the web, to the server software that generates websites, to the actual web site features themselves - JavaScript is core. And it is gaining features at incredible pace, which is why any of this is happening at all. I'm sure you won't have missed that this has been happening, but you may have been dismissing it for a while.

In my document-centric world best practice has always been that JavaScript is an enhancement you add to an already fully functional page. It was never something to rely on. It was never something to be used as a "main feature". And so, responsible web developers of the past quite rightly didn't treat it as a core leg of the stack. And quite rightly looked at early frameworks like Angular and Backbone and such with skepticism... and after a play, ignored them for "real work". That was not a wrong decision. At the time.

That attitude and approach toward JavaScript is becoming less and less a realistic best practice as time wears on - not because the benefits of progressive enhancement are no longer important, but because without the JavaScript, core features of the web stack don't work. Not just the fancy library like Angular or whatever we decided to ignore because of this reliance... core components of the web stack. The actual guts of the new web.

Web Components and CSS Houdini are two extremely powerful things that are happening, now. They alone are going to be game changers on top of what is already happening. And they do nothing without JavaScript.

Web Components allow you to write your own tags, which do whatever you want them to do. You get to write something like:

And have that render a tabbed interface, styled however you like, behaving however you like. The appeal is obvious. The idea is that libraries of these sorts of components can be made and your whole web page could be "composed" of them.

But without JavaScript, they're not going to do anything. The CSS Houdini project is in the same boat; it allows you to write your own CSS filters and effects, implemented via JavaScript.

Javascript has already become a fundamental requirement of one half of the web which deals in highly refined interaction instead of document presentation. And JavaScript is creeping into our core web technology features too.

Remember when you could build a website from nothing but hand coded files in a basic text editor? Still think that's the level of tooling that's appropriate for modern websites? Oh... you also think you deploy a website on a server these days? Hah. LAMP, how quaint.

Even ignoring the app side of things, JavaScript has come to dominate the tooling you need to spit our modern websites.

Complex tooling is something I am suspicious of - I do not like having complex or numerous dependencies; i believe that a boring stack is usually a better stack - so tools need to do a lot for me to decide they are worth learning, supporting, and the security exposure of relying on them.

Which is why I skipped the terminal based Gulp and Grunt "task runners" and stuck to something that took me far less time and did just as good a job for the jobs I needed - compiling basic SCSS, and optimising images. I ignored Browserify, and a bunch of here today gone tomorrow frameworks and tools. And I've not missed them. I've not dumped time into learning a dozen things that are already dead today, two or three years after they were "the cool new thing".

I use a GUI app called CodeKit for the SCSS to CSS steps. And for image compression? Content images are optimised server side, and design assets I manually optimise using other GUI apps or websites. I've never needed or wanted anything more complex or automated.

But, the rise of node and npm is relentless and they've had almost a decade now to mature and grow. And they have matured. And they are required by the new web development stack. And they do a ridiculous amount of work for you, if you can ever work out how to set it all up (sorry Andrew, you did your best but... ye gods what?).

Here's a pretty minimal list of appropriate tooling that can all apply now, whether you do simple document sites or full on web apps:

• node / npm - on your machine to fetch/build/compile stuff, or also on the server to do the same. Can also act as an Apache / PHP replacement.
• git - which is far from new and you ought to be familiar with, but which is where all the npm packages live, and where your web code should also live.
• webpack - It's used to compile the code you write into working deployable code that is the website. Because today we don't hand-author the code that runs in the browser...
• babel - because you won't write in the same JavaScript browsers understand, babel transforms the fancy modern JS you write into older JS even legacy browsers understand
• typescript - the web engineers at Microsoft hit limits of even the new JavaScript and so invented TypeScript... which compiles down into standard JavaScript. It helps ensure you right better JavaScript more efficiently.
• react / vue - the new "front end library" or "jQuery", only they're much more than that.
• A ton of sub-libraries that fit into the Vue / React ecosystem and provide more functionality such as state storage and routing.

Every one of these is a world unto itself. Every one will take a long time to learn their usefulness to your needs, and to "master". And these are just the raw technical things; not the best practices of each. Simply knowing which of these is a dependency on another is in itself a head fuck when you start out.

All of this is intimidating. And not all of this is required for the type of site I build. But, I've decided it's time to get on board with this stuff in preparation of further changes I can see down the line. So... how?

Well, over the coming year I'm wanting to tackle all of this, and I'm far too busy in my day job of building document-centric sites to learn on-the-job. So, it's going to be the odd free evenings. Here's my bucket list to tackle, in rough order of priority, though I know differing amounts about each already:

Basic Javascript

Start at the start. The JavaScript I know is old. Perfectly fine for what I do, but old. I know of arrow functions and spread operators etc, but not the what and the why. So... let's play catch up and learn plain JavaScript properly. It's the basis of literally everything else on the list.

I'll (once again) be working through Eloquent Javascript - a book I've bought and read twice now over the years, and which teaches the language itself. I will likely also try learnvanillajs.com as it's usually a good idea to have at least a couple of sources for similar things.

Specific client-side JavaScript features

There are one or two things Javascript can do on the website itself now, that I think are pretty important to get a website to do, and which can be done simply enough without resorting to the Rube-Goldberg technology stack of the full modern web developer experience:

Service Workers

These are small bits of code that run in the user's browser. They act as client-side proxies; which allow you to write some javascript that jumps in front of any outgoing https request and do something with it. Which means... you can have a website that does things even when the user is offline.

• Service Worker Fundamentals
  
• Mozilla's Service Worker Cookbook
• Workbox - a framework solution that may be useful after knowing the above two things.

Progressive Web Apps

This isn't strictly a "thing", nor JavaScript alone. It's a set of guidelines in best practices for a modern web-app or website, with Javascript requirements; and luckily enough any decent web developer will already be checking off a bunch of things. The "app" bit tends to be explained as 'you can install the website onto a phone or desktop' but that's likely the least useful or interesting thing about PWAs, in my opinion.

Progressive Web App Checklist

VueJS

Vue and React are both the "new" and "actually very well established by now" libraries within which most of the app-functionality of modern development is done. They are the frameworks of choice. They offer numerous benefits over trying to handle everything in plain JS because, as I'm sure you're aware by now, modern web apps do an awful lot. Vue and React abstract a lot of complications away into established battle tested and inter-operable practices.

I'll be going the Vue route instead of React, and that's because Vue is done in such a way that you can start using it inside normal document-centric websites using nothing but simple hand-written old-school script tags (yeah... those are "old school" now).

You can then move out to the more complicated but much more functional "stack of build steps in a nested javascript development environment" ... fun ... from there. Vue is the competent, established, functional, but easy to get started framework of choice. Vue is also community owned, incredibly popular, and not funded by the evil empire of Facebook. Which makes me happy.

The Vue documentation is some of the best I've seen since PHP's, so it's a good source of learning:

• Getting started with Vue - start where the official stuff says to start.
• Vue Style Guide - doing this early will help avoid common mistakes and anti-paterns.
• Vue Cookbook - because well explained real-world examples are worth their weight in diamonds.
• Vue Examples - quality not assured, but more examples are always good.
• Vue Forum - they have an offical forum. Now that is oldschool cool.
• Laracasts' Vue2 step-by-step guide - video tutorials that walk you through pretty well.

And once those 'basics' are at least well understood it'll be safe to explore the killer features of:

• The Vue CLI - a full on development stack for making Vue apps.
• Nuxt (and tutorials) - a framework in your framework *holds head in hands and cries*
• Vuex - a state management library for Vue applications.
• Gridsome - A Vue powered Static Site Generator.

Static Site Generators

What's old is new again. Way back in the early days we just uploaded plain HTML files we wrote by hand. Not long after tools were built to do that work for you, compiling stuff you wrote in a CMS to simple HTML documents that you then uploaded.

That's not been done in a long time, because people began wanting personalised pages - there's no way to do a shopping cart or a custom news feed that way. Except... now there is. And so there are a hoard of new projects that do this:

• Take content from markdown files, or a CMS, or any other parsable source
• Render every URL based on that content out to plain files
• Push those files out to CDNs

You don't even need a server now. These files are hosted on Content Delivery Networks close to your users, and because of that, and a complete lack of need for any processing on a server, they are insane fast to load, and super cheap to host.

What makes this practical these days is that interaction and user-customised data can be layered on top, because browser JavaScript is capable of that now. So, those flat files are delivered immediately, content is rendered immediately, and then the browser sends some API requests to a server and pulls only the bits of data that user needs, which is slotted into the resulting page. The only slow bits are the custom data bits, but there's less of that in every way. The whole way of doing this is very efficient.

There are tons of options here, but the main thing I want to learn is: getting CraftCMS to work smoothly as a headless client, so I can do this type of site deployment. Then figure out a little more about the JAM Stack if the above Vue bits haven't already answered that issue.

GraphQL

How does the client-side javascript talk to some server side data source, in order to populate additional or custom content to the page? Through an API, but the most modern method seems to be a new(ish) paradigm based on JSON interchange called GraphQL. It's a query language, it's essentially JSON as far as your client and server endpoints are concerned, and it looks incredibly convenient and powerful.

In a nutshell, you don't need to define exactly what data is sent to a client when a client asks for a pre-defined endpoint. The client gets to ask for specific data to be returned. You can, in theory, have a single end point on the server that does all the work you need. This, of course, will have consequences with regard to caching etc. But... looks like a strong and safe bet that GraphQL is going to be the way APIs are powered in future. So... lets learn that!

• Why GraphQL (vs REST etc)
• Graph QL - from the horses mouth
• Hooking up Craft with GraphQL - Craft is our CMS of choice, for very good reason. It works with Graph QL too.

node / npm

Hopefully you're familiar with apt or yum on Linux or brew on OSX, which you may have used to set up a web server in the past. npm is like that, but for your web projects themselves. Instead of going of to jQuery's website and downloading the jQuery file, you can grab it straight from npm. Along with everything else you need. Given that almost every bit of modern tooling pulls from npm... we need to know npm. Which means knowing a tiny bit of node too - because npm runs on top of node.

webpack

Honestly I wish I could avoid this. I've already sunk weeks into it and it remains almost impenetrable to me if only because of its odd concepts and painful configuration. It's used to hold almost everything else together. It can do things which I'm still not sure are worth the trouble - Hot Module Replacement, for example. Acting as a development server before building distributable code... it's a mess and a crutch IMO. But a needed one.

Theoretically it is most useful when you're building apps and not documents. It'll analyse your source code and create production code that is optimised in a variety of ways for the use cases you have. Right now, I've enough on my plate with it trying to get it to spit out compressed CSS with a source map to the file I want based on the source I want. It's a friggin' nightmare.

• Webpack

Deployment

Ok, so it's likely I will be sticking to the tried and true LAMP stack for a while. We can still get excellent performance out of that. However; it's always good to learn of alternatives. The more modern deployment stack doesn't usually use a VPS running LAMP. Instead it seems to be that you:

• Have your web project in a Master branch of a GIT repo
• Have a SaaS system watching for commits to that repo
• Have that SaaS system run or deploy a build from your GIT repo as it is updated
• Have that build pushed out to a CDN
• Have customisation handled via a traditional server acting as an API endpoint or... for real bleeding edge... hosted functions.

So, I want to learn the practicalities of this approach, and see whether it's a good fit for the type of work I do.

• Netlify

• Scrimba - 'video' tutorials but in a working code editor. Learn JS, Vue, Neural Networks, React... all sorts.
• Flaviocopes - Prolific content from a blogger. God I miss blogging.
• Mozilla Developer Network - they build browsers. They know their stuff. They want you to know their stuff.

I fully expect that some of this stuff just won't be applicable or worth it for the type of work I do. I'm in a small team and just don't require some of the things here which are aimed at solving problems that large distributed teams have, or that are for websites that have hundreds of thousands of visitors daily. But... it's always useful to know the rough landscape of things surrounding your problem area.