Introduction
I star a bunch of projects and find it very useful to browse through my stars, and see what I can pick up from various projects.
In this post, I will make a point to cover lesser known projects. While there’s lots to be learnt from, let's say, the React source code, I hope to share some some projects that you may not know about already.
Why and how I learn from other code
Sometimes, due to time constraints, motivation & simply personal preferences, one may prefer to absorb programming knowledge nuggets by reading existing code, as opposed to reading an entire book about some fundamental programming concept. I believe a good approach is using a combination of the two.
Often, learning new code techniques is a case of staring at it on my phone, maybe when travelling somewhere far. I’ll pick on something random in the codebase, such a ‘reset password’ feature within a Node.js project, and try to understand for example how the model + view + controller all support such a feature.
Reading code as an alternative to reading tutorials
Often, I’ll see large tutorials introducing or explaining a concept I want to learn about. I’ve written a few such tutorials myself, e.g. JavaScript streams! Tutorials can often include extra narrative, analogies, descriptions etc. making it harder to extract the minimal amount of code you need to try the feature out in the first place! This is where seeing the code can be more helpful.
Warning: Using this approach, you may miss out on important underlying principles which the author had taken the time to explain in the tutorial, such as gotchas and warnings as to when such a feature may not be the right approach. You may rush to see the usage of a new web platform API, and completely miss the fact that it's an experimental feature only supported in one browser!
Projects
I’ll limit this list to 9 projects, but if you enjoy the style & content, let me know and I’ll publish a part two.
➡️ Year Progress Twitter Bot in NodeJS
Link: MrDatastorage/Year-Progress-Twitter-Bot
This project creates a Twitter bot in Node.js which tweets out ‘year in progress’ updates, e.g.
What can you learn from this project?
For me, this focus isn’t necessarily on code quality, but rather: how to make a twitter bot and have it post at specified intervals. The code is pretty much all contained within one file, and has a number of comments to explain what is happening.
Posting a tweet
Being able to view config.js was also a useful reminder as to which configuration values I need to retrieve from Twitter in order to use the Twitter API.
<span class="hljs-keyword">const</span> T = <span class="hljs-keyword">new</span> <span class="hljs-title function_">twitter</span>(<span class="hljs-built_in">require</span>(<span class="hljs-string">'./config.js'</span>));
T.<span class="hljs-title function_">post</span>(<span class="hljs-string">'statuses/update'</span>, { <span class="hljs-attr">status</span>: bar});
Doing something at specified intervals from Node.js
In my personal side projects, I typically use setInterval, however this was a good reminder that cron jobs are a solid alternative especially when writing production grade code.
<span class="hljs-keyword">new</span> <span class="hljs-title class_">CronJob</span>(<span class="hljs-string">'0 0 * * *'</span>, <span class="hljs-function">() =></span> {
<span class="hljs-title function_">someFunction</span>();
}, <span class="hljs-literal">null</span>, <span class="hljs-literal">true</span>, <span class="hljs-string">'UTC'</span>);
➡️ A character count component created by gov.uk
Link: alphagov/govuk-frontend
This is to highlight a single ‘component’ in the gov.uk component directory. Examples may include: a table component, or what this post will look at, a character count component.
What can you learn from this project?
The high-level takeaway for me is: this is how gov.uk structures a frontend component for sharing between other teams. If I find myself having to create a reusable frontend component, I will certainly use this as a guide.
Live demos
Their readme links to a live demo of the character count component. Their demo page is simple and does not have gimmicky graphical elements that distract me from experimenting with the component. At a quick glance, I can see the HTML necessary to use this component. Their "When to use this component" and “When not to use this component” sections gives me confidence that this is a well thought out component that I can trust.
Configurability
I see they have a well defined schema of how the character count component may be configured. I feel that this in itself acts as documentation explaining how the component may be used.
Aria-live polite
I see they’ve used aria-live="polite" on an element which prompts me to read the ARIA Live Regions documentation on MDN.
Testing code with JavaScript disabled
I see their character count component test file and notice at least one thing in particular. They have a test which disables JavaScript in puppeteer (await page.setJavaScriptEnabled(false)) and then asserts the character count message displayed to the user. This acts as a guide for how to write my own tests for UI components, reminding me to disable JavaScript and make the necessary test assertions.
➡️ Third Party Decode
Link: paulirish/third-party-decode
This unmaintained project made interesting use of the Chrome DevTools JavaScript codebase.
What can you learn from this project?
Using the code of another project, even it wasn’t intended to be consumed.
Leveraging JavaScript functions from the Chrome DevTools source code
There are some require statements which look like this:
<span class="hljs-built_in">require</span>(<span class="hljs-string">'chrome-devtools-frontend/front_end/common/ParsedURL.js'</span>);
<span class="hljs-built_in">require</span>(<span class="hljs-string">'chrome-devtools-frontend/front_end/product_registry_impl/ProductRegistryImpl.js'</span>);
Notice they’re not being assigned to a variable, that’s because ProductRegistryImpl.js doesn’t export anything. It does however mutate an already existing property ProductRegistryImpl. To workaround this, the Third Party Decode project does this:
<span class="hljs-variable language_">global</span>.<span class="hljs-property">ProductRegistryImpl</span> = {};
While this may just be a workaround, it’s useful to see how the source code of Chrome DevTools, and other projects, can be imported into your own codebase.
➡️ Kick Them Out
Link: roccomuso/kickthemout
This project demonstrates the kicking out of other devices from your network by performing an ARP Spoof attack with Node.js.
What can you learn from this project?
As web developers, there’s a lot for us to keep up with. Occasionally, it’s useful to learn about something outside your speciality. In the past, network related tools like this were not written in JavaScript, and as a result, I didn’t invest any time learning how they work. The KickThemOut project uses JavaScript.
Checking the online status from node.js
Can’t comment on how robust this technique is, but a reminder that a DNS lookup is one way of determining internet connectivity:
<span class="hljs-built_in">require</span>(<span class="hljs-string">'dns'</span>).<span class="hljs-title function_">lookup</span>(<span class="hljs-string">'google.com'</span>, ...)
Good quality command line tools
A reminder on the importance of providing good quality tools, in this case, command line tools. There are a few things to note here:
<span class="hljs-variable language_">console</span>.<span class="hljs-title function_">log</span>(chalk.<span class="hljs-title function_">yellow</span>(<span class="hljs-string">' This task could take up to 5 minutes...'</span>))
- Colours are helpful to convey the state of the tool, e.g. yellow for an update of some sort, and green for success.
- The message in particular hints to the user how long the operation may take
- The line
new Spinner(' Updating MAC Vendors DB.. %s')creates a terminal friendly spinner
ARP Poisoning
As ARP poisoning is such a foreign concept to me, viewing it in a codified fashion makes it feel a tiny bit more approachable. At times like this, I remind myself that I do not understand the underlying theory of how it works - I simply know how to interact with an API to achieve a goal. I trace the arp.poison() call to a module called "arpjs" which performs the ARP packet poisoning:
arpPacket.<span class="hljs-title function_">send</span>({
<span class="hljs-string">'op'</span>: <span class="hljs-string">'reply'</span>,
<span class="hljs-string">'src_ip'</span>: ip2,
<span class="hljs-string">'dst_ip'</span>: ip1,
<span class="hljs-string">'dst_mac'</span>: <span class="hljs-string">'ff:ff:ff:ff:ff:ff'</span>
})
➡️ Lit Demos
Link: open-wc/lit-demos
Basic examples for working with lit-html and LitElement.
What can you learn from this project?
When encountering a new framework/library, I normally prefer to see minimal, straight to the point code examples without having to digest the project landing page, explanations, guides etc. It’s a personal preference however. Official project resources can sometimes be overwhelming in that you can’t find the exact thing you’re looking for, even it does exist.
In the case of this project, I appreciate small "cookbook recipes" (as they’re sometimes called), which demonstrates how to use and consume lit-html.
Fetching remote data and rendering it with lit-html
Immediately, when looking at it for the first time, I feel grateful I didn’t have to read lots of documentation to view certain techniques, e.g.
This defines your custom element:
customElements.<span class="hljs-title function_">define</span>(<span class="hljs-string">'fetching-data'</span>, <span class="hljs-title class_">FetchingData</span>);
You use static get properties() {} to define properties which are made available in the view.
Your custom element is a class, and it extends from LitElement:
<span class="hljs-keyword">class</span> <span class="hljs-title class_">FetchingData</span> <span class="hljs-keyword">extends</span> <span class="hljs-title class_ inherited__">LitElement</span>
As such, in the constructor, you can call super().
Handling JavaScript events with lit-html
I cmd + f on the page for ‘click’ making the assumption that this example demonstrates a click handler. I see it does:
<span class="hljs-comment"><!-- Use @[eventname] syntax to declaratively register inline event handlers --></span>
<span class="hljs-tag"><<span class="hljs-name">button</span> @<span class="hljs-attr">click</span>=<span class="hljs-string">${()</span> =></span> this.count += 1}>+<span class="hljs-tag"></<span class="hljs-name">button</span>></span>
Which tells me exactly what I wanted to know.
I remind myself that this is just one approach and if I wish to investigate further, I should then familiarise myself with the official documentation.
➡️ Prisoner categorisation - Ministry of Justice
Link: ministryofjustice/categorisation-tool
A tool to categorise prisoners in the U.K.
What can you learn from this project
It was a surprise to see code which may have traditionally been regarded as "sensitive", now be published out in the open. The prisoner data is kept private, but the web app which enables prison staff to categorise prisoners is available on GitHub.
This is personally interesting to me because it’s a domain which I’m unfamiliar with. For example, if you work for an online news website, you may be familiar with displaying large amounts of text and images on a page, and everything else that comes with that *domain *or landscape. In this case, how one categorises prisoners is a domain which is new to me and one which I was curious about.
Building a violence page
I browsed through some merged pull requests to understand how new features are developed. Just being exposed to the general workflow of how one would build a page which allows prison staff to assign a risk of violence flag to a prisoner was interesting.
Redacting sensitive data in logs
This served as a good reminder to myself, when logging information, ensure sensitive information is not logged and/or stored longer than necessary. Here, I notice they are replacing a session ID with a static string:
<span class="hljs-keyword">const</span> <span class="hljs-title function_">redactSession</span> = msg => msg.<span class="hljs-title function_">replace</span>(<span class="hljs-regexp">/session=[A-Za-z0-9=]+/</span>, <span class="hljs-string">'session=REDACTED'</span>)
➡️ Twitch Scripts
Link: noopkat/twitch-scripts
A number of scripts which GitHub user @noopkat uses to help automate their twitch streaming workflow.
What can you learn from this project
Learning from the automation scripts of people is always interesting since it may expose you to new automation techniques you wouldn’t have known otherwise.
Opening a set of Chrome tabs and positioning them (macOS specific)
This automation script is an applescript. It demonstrates how, given a set of websites, you can open each one up and position them accordingly.
repeat <span class="hljs-keyword">with</span> win <span class="hljs-keyword">in</span> chromeWindows
make <span class="hljs-built_in">new</span> <span class="hljs-keyword">window</span>
<span class="hljs-keyword">open</span> <span class="hljs-keyword">location</span> <span class="hljs-keyword">location</span> <span class="hljs-keyword">of</span> win
<span class="hljs-keyword">set</span> bounds <span class="hljs-keyword">of</span> front <span class="hljs-keyword">window</span> <span class="hljs-keyword">to</span> bounds <span class="hljs-keyword">of</span> win
<span class="hljs-keyword">end</span> repeat
Get the currently playing song from iTunes (macOS specific)
This line of applescript code stores the current track details in a variable.
<span class="hljs-keyword">set</span> song to run script <span class="hljs-string">"tell application <span class="hljs-subst">\"</span>iTunes<span class="hljs-subst">\"</span> to if player state is playing then <span class="hljs-subst">\"</span>now playing: <span class="hljs-subst">\"</span> & name of current track & <span class="hljs-subst">\"</span> by <span class="hljs-subst">\"</span> & artist of current track"</span>
➡️ Linguist
Link: github/linguist
GitHub performs language detection on source code which is then visually displayed to you through GitHub.com.
What you can learn from this?
Imagine if instead of whiteboard interview style questions, you were asked: "How would you generate source code language statistics, if you were a code hosting website?" Learning how this approach is handled elsewhere can expose you to new ideas.
How to generate language statistics
At a high level, the GitHub readme project explains. They seem to use a set of heuristics to make a best-guess attempt at which code language a source file is, e.g. using the file extension.
There are samples for different language types and a classifier.
How to model the configuration for all languages
GitHub appears to use a large YAML file for defining the configuration for languages. JavaScript starts like this:
<span class="hljs-attr">JavaScript:</span>
<span class="hljs-attr">type:</span> <span class="hljs-string">programming</span>
<span class="hljs-attr">tm_scope:</span> <span class="hljs-string">source.js</span>
<span class="hljs-attr">ace_mode:</span> <span class="hljs-string">javascript</span>
➡️ Negative Array
Link: sindresorhus/negative-array
This project supports using a negative array index to get items in backwards order, starting from the end, e.g. array[-1].
What can you learn from this?
MDN has some great documentation on JavaScript proxies, but also includes a lot of extra reading. This project utilises JavaScript proxies for a single use case, making the code useful to digest.
How to use JavaScript proxies to enhance language features
This negative-array project shows how a Proxy can wrap and intercept actions on a regular JavaScript array. The well named "target, name & receiver" method arguments on the get method also gives a useful hint as to what a Proxy getter has access to.
Good unit test practices
The test file demonstrates a few things:
Tests themselves can act as excellent documentation. In this case, the assertion t.is(fixture[-2], 'bar') lets me know exactly what should be possible with this module.
Unit tests can be well isolated: Here, I see four test functions which rely on hardly any global variables or even before/after callbacks which often rely on state.
Conclusion
If nothing else, I hope this post provided you with some new repositories you can take a gander at. The points I covered here are fairly small and don’t necessarily show larger architectural techniques. If you’d like to see such a thing, let me know on Twitter (@umaar) and I’ll cover that in a future post.