Creating a JWT secured React app and Kotlin server Part 4

In my last post I did the needful and installed code coverage to tell me if my unit tests were testing enough. You might argue that they might not be very good tests, and I’ve never said that coverage reporting addresses quality, but there are now nearly enough tests.

What I should do now is actually provide some proper functionality! The good thing about this, is we’ll be able to test it all properly too!

Database Entities

Now that we’re getting serious, let’s have some more entities. We’ll need to add Client and Code entities in addition to Users.

@Entity
class User(
        var username: String,
        var password: String,
        @Id @GeneratedValue var id: Long? = null)

@Entity
class Client(
        var name: String,
        var secret: String,
        var redirectUrl: String,
        @Id @GeneratedValue var id: Long? = null)

@Entity
class Code(
        var code: String,
        @ManyToOne var client: Client,
        @ManyToOne var user: User,
        @Id @GeneratedValue var id: Long? = null) 

Clients represent the different user interfaces that may seek to authenticate users, and Codes represent the one time codes generated by the Auth service to enable fetching a JWT. Clients should have a fixed redirect URL so that we can be certain we know where we’re sending the one time codes to. The client itself should provide this value as a kind of proof that it is the real deal. I’ve left out obvious properties here like the user’s name and other properties you might add, because our code coverage will report any fields that we’re not actually using as uncovered lines. Neat!

Note the use of @ManyToOne to link to a Code’s Client and User. Amazingly simple.

We’ll also want a couple of smart repositories to hold our queries.


interface UserRepository : CrudRepository<User, Long> {
    fun findByUsername(username: String): User?
}

interface ClientRepository : CrudRepository<Client, Long> {

}

interface CodeRepository: CrudRepository<Code, Long> {
    fun findByCode(code: String): Code?
} 

This is really neat. We’re using the magic methods from spring boot’s database engine to do simple queries, but by defining these interfaces we can specify how the implementation should work. My IDE (Intellij) will tell me if the interface I’m writing doesn’t match the actual implementation. For Clients we’re going to use findById which is a concrete method of CrudRepository so there’s no need for a function definition here. Now that we have these repositories we can inject them as dependencies with zero hassle!

And lastly we’ll add some initial data to our app, since this is all development work. (the database is only in-memory while the app runs!) See how easily we inject the repositories!

@Configuration
class AuthConfiguration {
    @Bean
    fun databaseInitializer(
            userRepository: UserRepository,
            clientRepository: ClientRepository,
            codeRepository: CodeRepository
    ) = ApplicationRunner {
        val user = userRepository.save(User("testuser", "testpassword"))
        val client = clientRepository.save(Client("Website", "secret", "https://localhost:3000/login"))
        codeRepository.save(Code("1234", client, user))
    }
} 

A JWT Service

Next, Let’s create a service to help us generate JWTs

we can use the https://fusionauth.io package to construct a JWT, add claims to it (the user ID and the client ID just now), and sign it with our secret (a little bit of a hack, let’s remember to use a config option for that later!)

        implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
+	implementation("io.fusionauth:fusionauth-jwt:3.5.3")
	developmentOnly("org.springframework.boot:spring-boot-devtools")
package com.chrisyoung.auth

import io.fusionauth.jwt.domain.JWT
import io.fusionauth.jwt.hmac.HMACSigner
import java.time.ZonedDateTime

class JwtService {
    fun createAccessToken(client: Client, user: User): String {
        val secret = "secret"
        val signer = HMACSigner.newSHA256Signer(secret)
        val jwt = JWT()
        jwt.addClaim("clientId", client.id)
        jwt.addClaim("userId", user.id)
        jwt.setIssuedAt(ZonedDateTime.now())
        jwt.setIssuer("auth-demo")
        return JWT.getEncoder().encode(jwt, signer)
    }
} 

Controllers

Now we want some functional controllers so we can go ahead and get that token from the front-end! We’re so close!

We’ll want to fill out the AuthController with the power to actually create those Codes

Our GET method should fetch the client from the database (and handle not-found scenario with an automatic response! Isn’t spring boot awesome!). We’ll add all the relevant information to the page for rendering the HTML (yes this route is traditional HTML rendered server-side)

@Controller
class AuthController(
        private val entityManager: EntityManager,
        private val clientRepository: ClientRepository,
        private val codeRepository: CodeRepository
) {
    private val chars = ('0'..'z').toList().toTypedArray()

    @GetMapping("/authorize")
    fun authorizeForm(
            model: Model,
            request: HttpServletRequest,
            @RequestParam(name = "state") state: String,
            @RequestParam(name = "clientId") clientId: Long
    ): String {
        val client = clientRepository.findById(clientId).orElse(null)
        client ?: return ":notfound"
        request.session.getAttribute("user") as User? ?: return "redirect:/login"
        model["title"] = "Authorize"
        model["clientId"] = clientId
        model["clientName"] = client.name
        model["state"] = state
        return "authorizeForm"
    }

Our form should display all the useful information to the user, and keep the relevant data in the hidden fields for the POST. Clicking the submit button is going to POST the data to the same route and create our secret code!

{{> _header }}
<div class="jumbotron">
    <form method="post" action="/authorize">
        <input type="hidden" name="clientId" value="{{clientId}}"/>
        <input type="hidden" name="state" value="{{state}}">
        <h2>Authorise {{ clientName }}?</h2>
        <br/>
        <input class="btn btn-primary" type="submit" value="Authorise">
        <button class="btn btn-danger" type="button" onclick="history.back()">Reject</button>
    </form>
</div>
{{> _footer }}

Our POST method should again fetch the client from the database, get the current user from the session, and create a new Code entity record before redirecting to the frontend with the string version of the Code.

    @PostMapping("/authorize")
    fun authorize(
            model: Model,
            request: HttpServletRequest,
            @RequestParam(name = "state") state: String,
            @RequestParam(name = "clientId") clientId: Long
    ): String {
        val client: Client? = clientRepository.findById(clientId).orElse(null)
        client ?: return "notfound:"
        val user = request.session.getAttribute("user") as User? ?: return "redirect:/login"
        val secret = (1..32).map { chars.random() }.joinToString("")
        val code = codeRepository.save(Code(secret, client, user))

        model["title"] = "Authorized"
        model["redirectUrl"] = format("%s?state=%s&code=%s", client.redirectUrl, state, code.code)
        return "authorize"
    }

The most significant parts here are the generation of the unique code (and saving to the database) and the creation of the redirect URL (if all goes well above) from the URL stored on the client record, the state value (a value provided by the client application at runtime representing the unique state at request time, which should be compared when receiving the code to prove the client application can trust the redirect request), and the one time code itself.

The rendered HTML just contains a javascript redirect to the url provided from the controller, and perhaps a nice UI to display while the user waits. You can tell I’m a back-end engineer as I’ve not bothered making it look nice.

{{> _header }}
Redirecting
<script type="text/javascript">
    window.location.href=decodeURI("{{redirectUrl}}")
</script>
{{> _footer }} 

And here’s some concise tests for the authorize form:

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class AuthControllerTests(@Autowired val restTemplate: TestRestTemplate) {
    @Test
    fun `Assert that the form is displayed`() {
        val entity = restTemplate.getForEntity<String>("/authorize?state=1234&clientId=2")
        assertThat(entity.statusCode).isEqualTo(HttpStatus.OK)
    }
    @Test
    fun `Assert that the redirect happens if not logged in`() {
        val entity = restTemplate.postForEntity<String>("/authorize?state=1234&clientId=2")
        assertThat(entity.statusCode).isEqualTo(HttpStatus.FOUND)
    }
    @Test
    fun `Assert that wrong clientId fails`() {
        val entity = restTemplate.getForEntity<String>("/authorize?state=1234&clientId=1")
        assertThat(entity.statusCode).isEqualTo(HttpStatus.NOT_FOUND)
    }
    @Test
    fun `Assert that wrong clientId fails POST`() {
        val entity = restTemplate.postForEntity<String>("/authorize?state=1234&clientId=1")
        assertThat(entity.statusCode).isEqualTo(HttpStatus.NOT_FOUND)
    }
    @Test
    fun `Assert that the redirect happens`() {
        val loginResponse = restTemplate.postForEntity<String>("/login?username=testuser&password=testpassword")
        assertThat(loginResponse.statusCode).isEqualTo(HttpStatus.OK)
        val sessionCookie = loginResponse.headers.get("Set-Cookie")?.get(0)?.split(';')?.get(0);
        val headers = HttpHeaders();
        headers.add("Cookie", sessionCookie)
        val entity = HttpEntity<Any>(headers)
        val result = restTemplate.exchange<String>("/authorize?state=5678&clientId=2", HttpMethod.POST, entity)
        assertThat(result.statusCode).isEqualTo(HttpStatus.OK)
    }
}

Of course we should add some more assertions, but this just demonstrates how easy it is to perform the test.

Next, our TokenController will be a rest endpoint that will actually give us the JWT we’ve been waiting for! All the controller will do is fetch the code from the database (or show not found error) and use the JWT service to create a JWT and return a JSON response with the accessToken and (still dummy) refreshToken and the info about the user.


data class TokenRequest(
        val code: String
)

data class TokenResponse(
        val accessToken: String,
        val refreshToken: String,
        val user: User
)

@CrossOrigin(origins = ["http://localhost:3000"])
@RestController
class TokenController(val codeRepository: CodeRepository) {
    @PostMapping("/token")
    fun createToken(@RequestBody(required = true) tokenRequest: TokenRequest): ResponseEntity<Any> {
        val code = codeRepository.findByCode(tokenRequest.code) ?: return  ResponseEntity.notFound().build()
        val token = JwtService().createAccessToken(code.client, code.user)
        return ResponseEntity.ok().body(TokenResponse(token, "refresh-token", code.user))
    }
}

Note the use of the @CrossOrigin annotation to allow browser AJAX requests from different domains (in local development from a port 3000, React’s favourite), and the use of @RestController rather than just @Controller. This does some nice magic around providing a JSON response.

I quite enjoy the easy definition of what a request and a response look like, so that we can strictly type the controller method. A Token Request needs only the one-time code, and a Token Response will only have the JWT (access token), a refresh token, and the user object so we have some info about the user in the response body. We’ll use that later to display a welcome message.

No templates here, it’s a REST controller! Here’s a little test though:

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class TokenControllerTests(@Autowired val restTemplate: TestRestTemplate) {
    @Test
    fun `Get a token successfully`() {
        val headers = HttpHeaders()
        headers.contentType = MediaType.APPLICATION_JSON
        val request = HttpEntity(TokenRequest("1234"), headers)
        val entity = restTemplate.postForEntity<String>("/token", request)
        assertThat(entity.statusCode).isEqualTo(HttpStatus.OK)
    }
    @Test
    fun `Bad code`() {
        val headers = HttpHeaders()
        headers.contentType = MediaType.APPLICATION_JSON
        val request = HttpEntity(TokenRequest("wrong"), headers)
        val entity = restTemplate.postForEntity<String>("/token", request)
        assertThat(entity.statusCode).isEqualTo(HttpStatus.NOT_FOUND)
    }
}

Again only testing the response codes, but roughly I know I haven’t broken the controller entirely.

Let’s also add a Verify endpoint so the client application can check the JWT is (still) valid:

@CrossOrigin(origins = ["http://localhost:3000"])
@RestController
class VerifyController {
    @GetMapping("/verify")
    fun verifiy(
            @RequestHeader(name = "Authorization") auth: String
    ): JWT? {
        val token = auth.replace("Bearer ", "")
        return JwtService().verify(token)
    }
}

I’ll also throw in Twitter Bootstrap for some slightly Nicer styles:

<html>
<head>
    <title>{{ title }}</title>
+    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css" integrity="sha384-JcKb8q3iqJ61gNV9KGb8thSsNjpSL0n8PARn9HuZOnIxN0hoP+VmmDGMN5t9UJ0Z" crossorigin="anonymous">
</head>
<body> 

Here’s the login screen:

Screenshot from 2020-10-30 18-07-40

And the Authorize screen:

Screenshot from 2020-10-30 18-09-38

My code coverage is doing alright too

That'll do for now

That’ll do for now

In the next blog, I’ll show you the react app I’ve created to log in using these routes.

Creating a JWT secured react app and Kotlin server (part 3)

Build tools, Unit tests, and Code coverage

The base application I generated from the Spring Boot Initializer was a Maven project. I had the option of Maven or Gradle, and I made a quick choice based on my knowledge at the time. You see, I’m a PHP developer. I’m used to there being one dominant package manager, Composer, and not having to make choices about which one to use. My experience with Gradle and Maven has been that Gradle is for Android projects and Maven for Server-side things. It’s not actually that clear cut, it’s just that Google has chosen Gradle to be the official build tool for Android, and maven is the original package manager so older projects run on maven.

I did experience a little bit of this package manager duality when developing JavaScript applications using either the Node Package Manager, or the stand-in replacement Yarn, developed by Facebook’s React team because npm was too slow, but yarn used the same package.json file as npm, so switching between the two only meant losing your package lock file.

Yarn’s biggest claim to fame is speed, and it seems that is what Gradle brings to the table also. Again, as a PHP developer I’m used to my code being interpreted at runtime, rather than being compiled with every code change, and it hadn’t even entered my mind that build speed would be a factor, but golly was I bored watching my application build over and over again to fix simple mistakes I’d made in my code.

The gradle website presents a very convincing demonstration of how much faster Gradle builds are, and that it actively avoids doing unnecessary work. It sings the praises of it’s “work avoidance mechanism”, and shouldn’t we all? I was sold on the promise of faster builds, and the result didn’t disappoint. Have a look here https://gradle.org/maven-vs-gradle/ if you’re interested in more reliable information.

Converting

I mentioned earlier the conversion from npm to yarn for an existing project was as simple as running `yarn` instead of `npm i`, but it’s not like that for Gradle, it’s a totally different structure. Maven has it’s pom.xml file that looks a bit like this

Maven pom.xml

And Gradle has a syntax that reminds me a bit of json and a bit of yaml. Apparently it’s a superset of Java and it’s called Groovy.

plugins {
    id 'application'
    id 'groovy-base'
}

dependencies {
    implementation project(':utilities')
    testImplementation 'org.codehaus.groovy:groovy-all:2.5.11'
    testImplementation 'org.spockframework:spock-core:1.3-groovy-2.5'
}

application {
    mainClass = 'org.gradle.sample.app.Main'
}

The thing is that now with the introduction of Kotlin, everyone’s moving from Groovy DSL to Kotlin DSL which uses Kotlin syntax. I guess this is better, but what it means is that documentation is completely fragmented between maven XML, and the two different Gradle formats, and if you ever ask someone for help there’s a 66% chance they’ll tell you in some other syntax and you’ll have to covert it.

plugins {
    application
    `groovy-base`
}

dependencies {
    implementation(project(":utilities"))
    testImplementation("org.codehaus.groovy:groovy-all:2.5.11")
    testImplementation("org.spockframework:spock-core:1.3-groovy-2.5")
}

application {
    mainClass.set("org.gradle.sample.app.Main")
}

Apparently if you just run gradle in a maven repo, Gradle will do the coversion for you. Unforunately this creates a Groovey DSL, and all the kotlin docs give you instructions in Kotlin DSL, plus we need different plugins for Gradle, so it was easier in the end to re-generate the base project from the Spring Boot Initializer and copy in the new files. I ended up with this commit and this DSL

import org.jetbrains.kotlin.gradle.tasks.KotlinCompile

plugins {
	id("org.springframework.boot") version "2.3.3.RELEASE"
	id("io.spring.dependency-management") version "1.0.10.RELEASE"
	kotlin("jvm") version "1.3.72"
	kotlin("plugin.spring") version "1.3.72"
	kotlin("plugin.jpa") version "1.3.72"
}

group = "com.chris-young"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_11

repositories {
	mavenCentral()
}

dependencies {
	implementation("org.springframework.boot:spring-boot-starter-data-jpa")
	implementation("org.springframework.boot:spring-boot-starter-mustache")
	implementation("org.springframework.boot:spring-boot-starter-web")
	implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
	implementation("org.jetbrains.kotlin:kotlin-reflect")
	implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8")
	developmentOnly("org.springframework.boot:spring-boot-devtools")
	runtimeOnly("com.h2database:h2")
	testImplementation("org.springframework.boot:spring-boot-starter-test") {
		exclude(group = "org.junit.vintage", module = "junit-vintage-engine")
	}
}

tasks.withType {
	useJUnitPlatform()
}

tasks.withType {
	kotlinOptions {
		freeCompilerArgs = listOf("-Xjsr305=strict")
		jvmTarget = "11"
	}
}

Code coverage

For code coverage I want to use a service called coveralls. I have chosen a Gradle plugin  that will upload reports to that service. To generate those reports we’ll use a Gradle plugin that produces coverage reports from unit tests called Jacoco.

To add the coverage report generator, we simply need to add the plugin here

plugins {
   id("org.springframework.boot") version "2.3.3.RELEASE"
   id("io.spring.dependency-management") version "1.0.10.RELEASE"
   id("java")
   id("jacoco")
   ...
}

The jacoco test report comes out looking like this.

Jacoco test report

Jacoco test report

This HTML report is very handy for local development – Next we need to enable XML report generation for export into coveralls.

tasks {
   jacocoTestReport {
      reports {
         xml.isEnabled = true
      }
   }
}

And then we’ll add the library to export to coveralls. Because this is a public repository, there’s no configuration required.

plugins {
   ...
   id("jacoco")
   id("com.github.kt3k.coveralls") version "2.10.2"
   ...
}

Lastly we’ll add a travis file (.travis.yml) to enable CI builds.

language: java

jdk:
- oraclejdk11

cache:
  directories:
    - $HOME/.gradle/caches/
    - $HOME/.gradle/wrapper/

install:
- ./gradlew assemble

script:
- ./gradlew test build

after_script:
  - ./gradlew jacocoTestReport coveralls

Theoretically all that’s needed is language: java and travis will automatically detect the build.gradle file and use gradle to build and test the  repo, but since we want to use a newer version of Java and Gradle than the ones that ship with travis, and we want to do some specific step at the end, it’s better if we are a bit more specific about all the build steps.

Now when we push to GitHub, Travis CI will run the build script and upload the test reports to Coveralls.

resolvconf
install_jdk
Installing oraclejdk11
0.00s
git.checkout
0.44s$ git clone --depth=50 --branch=main https://github.com/darkbluesun/auth-demo-auth.git darkbluesun/auth-demo-auth
0.01s$ export TERM=dumb
cache.1
Setting up build cache
$ java -Xmx32m -version
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)
$ javac -J-Xmx32m -version
javac 11.0.2
install
21.04s$ ./gradlew assemble
25.22s$ ./gradlew test build
> Task :compileKotlin UP-TO-DATE
> Task :compileJava NO-SOURCE
> Task :processResources UP-TO-DATE
> Task :classes UP-TO-DATE
> Task :compileTestKotlin
> Task :compileTestJava NO-SOURCE
> Task :processTestResources NO-SOURCE
> Task :testClasses UP-TO-DATE
> Task :test
2020-09-13 02:10:57.345  INFO 4737 --- [extShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2020-09-13 02:10:57.345  INFO 4737 --- [extShutdownHook] .SchemaDropperImpl$DelayedDropActionImpl : HHH000477: Starting delayed evictData of schema as part of SessionFactory shut-down'
2020-09-13 02:10:57.356  INFO 4737 --- [extShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2020-09-13 02:10:57.358  INFO 4737 --- [extShutdownHook] .SchemaDropperImpl$DelayedDropActionImpl : HHH000477: Starting delayed evictData of schema as part of SessionFactory shut-down'
Hibernate: drop table if exists user CASCADE 
Hibernate: drop sequence if exists hibernate_sequence
2020-09-13 02:10:57.587  INFO 4737 --- [extShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'
2020-09-13 02:10:57.592  INFO 4737 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2020-09-13 02:10:57.623  INFO 4737 --- [extShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2020-09-13 02:10:57.625  INFO 4737 --- [extShutdownHook] .SchemaDropperImpl$DelayedDropActionImpl : HHH000477: Starting delayed evictData of schema as part of SessionFactory shut-down'
2020-09-13 02:10:57.635  INFO 4737 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.
2020-09-13 02:10:57.638  INFO 4737 --- [extShutdownHook] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'
2020-09-13 02:10:57.639  INFO 4737 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-2 - Shutdown initiated...
2020-09-13 02:10:57.645  INFO 4737 --- [extShutdownHook] com.zaxxer.hikari.HikariDataSource       : HikariPool-2 - Shutdown completed.
> Task :bootJar UP-TO-DATE
> Task :inspectClassesForKotlinIC UP-TO-DATE
> Task :jar SKIPPED
> Task :assemble UP-TO-DATE
> Task :check
> Task :build
BUILD SUCCESSFUL in 24s
6 actionable tasks: 2 executed, 4 up-to-date
The command "./gradlew test build" exited with 0.
before_cache.1
0.00s$ rm -f  $HOME/.gradle/caches/modules-2/modules-2.lock
before_cache.2
0.00s$ rm -fr $HOME/.gradle/caches/*/plugin-resolution/
cache.2
store build cache
after_script
11.19s$ ./gradlew jacocoTestReport coveralls
Done. Your build exited with 0.

Once the coverage report is uploaded it looks like this:

Coveralls code coverage

Coveralls code coverage

Here’s the full commit on Github.

Next I’ll add some actual functionality and get that sweet 100% coverage.

Creating a JWT secured react app and Kotlin server (part 1)

‘ve been a PHP software developer for as long as I’ve been developing software. My first taste of programming in university was in PHP, and whilst I’ve developed in Java and Javascript at different points in my career – I do a fair bit of javascript at the moment actually – I’ve always come back to PHP. So since change is like a holiday, I wanted to take some time to learn some new things that are rising in popularity. Furthermore as I journey into the micro-services space, I’m dealing daily with Javascript Web Tokens (or JWTs) to authenticate and authorise requests in my PHP services, and I’m also dealing with their creation.

For this project I wanted to capture the handing of authentication and authorisation by building an auth server, a resource server, and a frontend. The SPA frontend will have a landing page, a login function which will redirect to the authorise page on the auth server, which in turn will redirect back to the frontend with a one time code. The frontend will then fetch an auth token (JWT) using the one time code and client credentials. Finally the frontend will request resources from the resource server using the auth token.

For my two backends I wanted to try something completely new. I hadn’t written anything in Java for a very long time, and I’ve heard very good things about Kotlin since it’s introduction. Kotlin is a superset of Java, adding features many consider to be sorely needed such as strict typing and null handling. I’ve seen it used in Android and desktop applications, but also in APIs. I’ll write both my backend APIs (the auth server and the resource server) in Kotlin with the Spring framework, which appears to be the most popular.

Kotlin Programming Language

Whilst it would be possible to write my frontend in Kotlin also, I wanted to explore a more popular web application framework, React. I’ve used react, and React Native a fair bit recently, however developers with a front-end focus have spent more time than I on those projects. I also used Redux heavily last time, and I’d rather try to avoid that by using React hooks instead. I’ll write my react app in Typescript, since I highlighted that strict typing was one of the reasons for choosing Kotlin above.

Getting started

Source control

Where am I going to put this code that I’m writing? In a git repo of course! And I prefer github.com for its ease of use and many features. I toyed with the idea of keeping all three parts in the one repository, given that they’re part of one (fairly limited) project, but the way that tools like travis and coveralls work, each thing really needs it’s own repository. Good thing they’re unlimited!

Initialisation

As a PHP developer, I’m accustomed to using composer to spin up a project, and in a similar way using npm or yarn to initiate a javascript project. This project starts a little differently. Since I’m using Spring Framework, the recommended way to proceed is to download a zip file with your starter project inside. Alright, let’s do that!. I went to https://start.spring.io/ to build a starter package. (First I did some language tutorials at https://kotlinlang.org/, but I’ll let you imagine that part).

Spring boot initialiser

Spring boot initialiser

 

I stuck the unzipped files in a folder called auth inside my project folder.

Java Versions

My mac comes with Java 8 / 1.8. That’s not good enough for this project, so I needed to get homebrew to download and install Java 11.

brew tap homebrew/cask-versions

brew cask install java11

Starting the react project was a bit more of a familiar project. I was accustomed to having the latest react-native-cli installed globally with NPM, but now you just run

npx create-react-app frontend

It’s truely a beautiful process, though admittedly the product you get in the end is much lest customised.

Create React App

What IDE am I going to use?

A few years ago I got really attached to Jetbrains PHPstorm for php development. The full featured IDE was everything i needed for debugging testing and writing quality code. Of late working on both PHP and Javascript projects I had taken to using vscode for everything. The PHP plugins provided me most of the IDE-type functionality I desired, and the javascript functionality was out-of-this-world. Since I’ll use vscode for the frontend, I assumed that I could continue to use vscode for Kotlin development too, and certainly there was a fair bit of Java and Kotlin support in the community and in the plugin repo, but ultimately I found it unworkable and decided to switch back to Jetbrains IntelliJ IDEA which, thankfully, is free (that is, the community edition is all I need for this project).

I’m very impressed with the experience of IntelliJ IDEA. Especially given how foreign the Java ecosystem is to me at this point. Managing build, dependencies and run is a lifesaver.

IntelliJ IDEA

IntelliJ IDEA

Of course things weren’t perfect at first – until this popup appeared to save the day

Non-managed pom.xml file found

Adding it meant that the IDE installed all the dependencies for me, and could auto-import.

Screen Shot 2020-08-30 at 2.17.32 pm

Next, I’ll write some actual code. Go to part 2

Android

Screenshot 2017-04-24_20-47-12

I decided I’ve been putting off learning Android development for far too long. I’ve completely avoided the idea like it’s some fad that’s gonna pass when, you know, everyone ditches smartphones. My lack of motivation to try the platform was my absolute lack of ideas as to what to develop. I’m just not one of those people who come up with cool ideas for apps.

Recently though, my personal outrage at a hideous app boiled over and a google search for a public API for the service actually yielded some result. The API doesn’t do exactly what an API should, which kind of explains why the app is so gosh darn awful, but it does work.

The first thing that I discovered when I downloaded and installed Android Developer studio on my mac, was how amazingly similar the interface is to PhpStorm by JetBrains, which I use for developing an app in Symfony2 at work. Obviously they’re based on the same IDE framework, but it really was a shock – they’re almost identical.

Screenshot 2017-04-24_20-46-32

The next delightful surprise – remember I’d stuck my head completely in the sand – was that apps are written in Java. Java! I haven’t touched Java since my university days. I’d have to say that it doesn’t seem like a whole lot has changed with Java, but so much with PHP has changed that the two have come a lot closer together. Honestly, it doesn’t feel that different to programming in PHP except, well, it’s not PHP.

On my workplace’s website I’m listed as a “full stack developer” which means I can’t decide what part of the stack to specialise in so I just try to be reasonable at everything. I’ve taken to programming frontends in angular, which has given me a much more up to date understanding of Javascript for frontend development – but this is Java, and it really bears no resemblance to me. Java feels more like PHP, but I’m writing *frontend* code in in, which just feels all wrong. Nevertheless the IDE, the tutorials and good old Stack Overflow practically guided me through the experience in a couple of days.

Screenshot 2017-04-24_20-46-57

The next surprise – again, I had no idea what was in store – was that I didn’t have to code in that gosh darn Hyper Text Markup Language that I’ve grown to hate. Instead, there’s a cool visual editor that helps you lock everything in place – and it’s not wishy washy about how it works. You can easily view the XML that it outputs, and adjust it yourself if you like that kind of thing. One thing I can’t do, is make anything look any good – so if there’s a drag and drop editor that let’s me assign actions to buttons, then awesome.

As I said, I found the public facing API documentation on the website for the original app. The system honestly scares me. Authentication for the API is either via an API key which you can only acquire by contacting customer support – and only if you’re the owner of the organisation, not an end user. The alternative is to send your username and password in the query string of every single request – which means that the app has to store your password. When you call the Authenticate endpoint – the system does give you a token, but there’s absolutely nothing in the API documentation about how to use that token to authenticate. What shocked me, is what happens when you stick in invalid credentials. The request returns successfully with a 2xx status code, and a user object with null fields. You have to check if one of the fields is null (and it could be null for other reasons) to detect that authentication has actually failed. Crazy.

So that’s what happens if you send the right fields with the wrong values. Guess what happens when you send the wrong fields? The API documentation says that the default format for responses is JSON, and that you can request XML if you so desire (because you’re insane) and for the most part this is true – but if you happen to not send a mandatory field, or some invalid value that causes an internal server error – you get a HTML Page with the error on it. Just let that sit for a minute. If I want to tell the user why the request failed (admittedly because I malformed the request) then I have to somehow interpret their HTML 500 error page and jam it onto the little phone screen. All I’ve done so far is just dump the raw html onto the screen, because I’m just not touching that.

I recently had to design and implement an API for a SAAS and making sure the app returns the correct status codes and errors in the correct format was possibly more important than getting the API to actually do anything. I honestly can’t see how they missed this.

In any case I had a little bit of fun, and that’s what I set out to do. I might have a go at getting the app good enough to put on the app store – and there are quite a few more features that I’d like to add to it, but I’m pleased that in a couple of days I made an app that functions better than the one my company pays a monthly fee to have access to – even though a couple of days ago I didn’t have a clue how to make an app.

You can check it out here.

https://github.com/darkbluesun/nimbleschedule-android-timeclock

Hearing is believing

I’m an amateur sound guy. On the weekends I do the sound for a church with a small band – Drums, Bass, Acoustic guitar, a couple of vocals, and electric guitar on a good day. I know as much as I need to know to keep the band and the congregation happy. I’ve never studied audio. I listen to MP3s and I don’t like records.

So when I was invited to a Digital Console Showcase in a nearby church where they would be comparing the sound quality of the mixers in a blind test – I figured I would barely notice any difference, unless the quality of a mixer was really bad. Well, it turns out that I can hear the difference between the mixers – and that my observations were pretty much on par with the group. I’m on the lookout for a 16 channel digital mixer.

Here’s what we did. We had a set wave file with lots of different types of music and speech in a short clip. We had each listened to it beforehand. Then, the file was played at nominal gain with no tuning through five different desks: the Roland M200i, the Behringer x32, Midas m32, Yamaha QL5, Soundcraft Performer SI2, and Allen & Heath’s GLD 80 – though only the operator knew which mixer was number 1, 2, 3, 4 or 5. Predictably nobody really noticed a difference between the desks (or at least consistently across the group) and neither did I.

Soundcraft SI performer 1

X32-COMPACT_P0AAP_Top_XL

Behringer x32 compact

m200i_top_main_gal

Roland m200i

QU16_Front_Main

Allen & Heath QU16

The next step was to play with the EQ. The sound file was then played through the mixers with different eq settings. +10dB @5k Q1, +10dB @1k Q2 and finally +10dB @150 Q1. We listened to see which mixers gave the most musical or transparent results for the increase in gain. At this point i started to notice differences in the sound between the mixers. I could swear that the EQ change on the first mixer had a much harsher effect. In the 1K test the fourth desk sounded awful – and I knew by the reactions of those around me that I wasn’t wrong. Sadly, we were told later that there was an error with that mixer in that test.

Lastly we listened to the track with heavy compression. 5:1 Attack 15mS Rel 100mS reducing 10db. In this test I could definitely hear that some mixers just didn’t do a great job of compressing the sound without altering it.

The results were surprising to me. For example the Behringer x32 performed quite well – about on average with the other mixers – proving that the inclusion of Midas pre-amps with the mixer have made a big difference given the company’s less-than-satisfactory reputation for audio quality. That said, most people (except me) noted it sounded poor in the compression test. Maybe that’s something I need to listen out for more.

No desk stood out particularly above any other, and (if you exclude the botched 1k test results) no desk stands out as particularly poor either. Here they are:

Screenshot 2014-11-28 10.34.30

As a point of interest we listened to the sound file played through all 5 mixers with the gain pushed to solid red. Needless to say, they all sounded bad, and it wasn’t really a very measurable test anyway.

Opencart 2 and GST (or other taxes) included

Have you seen the new Opencart version 2? It’s a really nice re-build of the ageing 1.x version of the simple open source shopping cart. If you’ve decided you don’t want the hassle of trying to tame Magento to your needs, you’ll notice that it’s a whole lot simpler. Unfortunately the developers have left out a critical feature in my mind. You must enter a product price (and any discounts, options, specials, etc) excluding any taxes. Tax – as configured by the store set-up, and applied according to the location of the buyer – is then applied to the product for display on the website and in the cart/checkout. If you our your client are selling products locally or nationally, but not internationally this might seem like a bit pain in the behind. Frankly I can’t understand how they would leave such a crucial piece of functionality out of the code base.

admin no taxfrontend with tax

Opencart 1.x is notoriously difficult to extend, beyond a few simple functions like payment gateways and shipping methods. Fortunately the community developed vQmod. This add-on makes a few changes to the core files, and then uses find-and-replace XML files to create modified versions of the core code and uses them instead of the originals. You can get vQmod files to change just about any functionality in OpenCart 1.x.

In Opencart 2, the developers have included their own system for rewriting the code base called OCMOD. I’m sure you can guess what it stands for. It works on exactly the same principal, although there was some disagreement on the way the XML files should be structured so the two are not strictly compatible.

I’ve written a simple OCMOD XML file to add some new fields to the admin section of Opencart. It’s based on a vQmod XML file for Opencart 1.x. The extra fields simply allow you to enter a price including your configured tax rate, and the ex tax price is calculated instantly using Javascript. Only the ex tax price is saved in the database. This mod will work with the base price, options that add or subtract from the price, qty discounts and specials.

You can download it free here:

http://www.opencart.com/index.php?route=extension/extension/info&extension_id=19744

You can contribute to the code here:

https://github.com/darkbluesun/oc2-prices-inc-tax

Screenshots below:

input-price-including-taxesspecialdiscount

option

Laravel generates a system fast

Today I showed off a task management and time tracking system I’ve been working on this week. The system isn’t unique or marketable, but simply meets the organisations needs without incurring a monthly fee – as so many systems do nowadays.
Not only did my co-workers and managers think it was a great system, they also were astounded at how quickly I’d produced a fully functioning system. The answer is with Laravel, generators and Twitter Bootstrap.

Laravel

There are so many good frameworks out there today that nobody should ever code anything in straight PHP. When you use a framework, you have access to code libraries that take care of the basic functions of a web application – routing, security, authentication, database abstraction and more – for you, better than you would handle them yourself. I’ve found Laravel to be an exceptionally easy framework to use.

My application didn’t need to focus on authenticating users, writing SQL statements, dealing with URLs and fighting cross-site scripting and SQL injection. I just got on with the business logic of my application.

http://laravel.com/

Generators

Whilst Laravel itself does not come with particularly powerful code generation – perhaps that’s not in the scope of a framework – Jeffrey Way has developed an outstanding set of generators for the Artisan command line interface. These generators help you develop anything from a simple database migration to a model to a whole set of code to support an object. Say I have an application to manage pets. Generate a pet scaffold with the generators and you’ve got a database migration, a model, a controller and a set of views. Everything you need to get started developing and avoid time-wasting boilerplate writing.

Knowing the objects I needed to work with, I simply generated all the scaffolds and then got to work coding the specifics.

https://github.com/JeffreyWay/Laravel-4-Generators

Twitter Bootstrap

I am not a designer. I don’t want to spend my time designing. I used to create really ugly admin panels. Not any more. Twitter Bootstrap allows you to create a beautiful user interface without trying. Just include the CSS and JS files in your source, and use the CSS classes to style your application. It even comes with an awesome icon font to give you lovely buttons instead of ugly links for your actions.

http://getbootstrap.com/

With these three tools, I could develop an application extremely fast, yet develop code that won’t have the next developer hunting me down for revenge.

 

 

Adding a Boolean Toggle to a grid in Joomla admin

Joomla has a fantastic tutorial on MVC components on it’s website, but it seems to stop way short of explaining how to make a fully featured admin panel.

One of the features it doesn’t seem to explain well is how to add those little green ticks and red circles that you can click and it changes (toggles) from one to the other.

Joomla Demo - Administration - Google Chrome_2013-04-30_16-55-57

I like to use it for chosing if an item will appear on the front page or not – but I’m sure you could think of other uses for it. Joomla also has a built in ‘Published’ button which is a bit simpler, so I’ll start from that and move on up. Continue Reading…

Hadrian’s wall

DSC03472

This is the second weekend in a row we’ve managed to get out and explore the beauty of the world around us, and I’m loving where God has placed us in the world. We walked along a picturesque section of Hadrian’s wall – like the Great wall of China, but not quite as great. It was built with much the same purpose: to keep out the northern barbarians – in this case the Scots. Continue Reading…

Invisible roadworks

2013-02-09 16.35.08

In Australia roadworks were traditionally done by the local council, and though for many years it has been subcontracted out to private companies there is still a general image of ‘council workers’ who dress in reflective gear and sit by the roadside smoking cigarettes. I’ve discovered recently that it’s the same in England – except for the name.

All around the world I have seen this same phenomenon. I can’t imagine why the government needs to install traffic cones along a 3 mile stretch of the motorway where there is no appearance of any work being done or any damage to the road. Do they just do it for fun?

I decided that I needed to look this up on the web. The Highways Agency twitter account mentioned the closed lane:

#M6 northbound between J39 and J40 | Lane(s) blocked http://bit.ly/sutXS1

Still, I found it very difficult to find out WHY the lanes were blocked – as the highways agency doesn’t allow you to search in the past (though it’s helpful for the future – http://www.trafficengland.com/disruptions.aspx)

Eventually I found this on a cached version of the advanced driving website:

http://www.advanced-driving.co.uk/traffic-reports/north-west-england/

M6 northbound between J39 and J40 | Northbound | Resurfacing

On the M6 northbound between junctions J39 and J40, minor delays can be expected at peak times due to carriageway resurfacing , between 8 pm and 6 am, from 5 February 2013 to 8 February 2013.

I passed through on the 9th of February, so I guess they finished the job and decided to leave the cones in place one more day (or more!). Oh well.

Yesterday I saw some guys doing works on the road while there was snow coming down and wind blowing it everywhere, so I guess that kind of makes up for it.