reading, writing, rambling

Resources for New Managers

Wed, 09 Mar 2022 00:00:00 GMT

I haven’t written much over the last two years because I have been pretty steeped in the adventure of becoming an engineering manager. My organization has more than doubled in size over the last year, and I am now in the position of mentoring for other folk who are beginning their journey on the manager track. I realized that I have read a significant number of management and leadership books over the last few years, so I put together a list of my favourites that have helped me most along the way.

The Manager’s Path — Camille Fournier

Absolute MUST READ for anyone at the IC or Tech Lead level who is curious about entering leadership. Camille shares her journey from Senior Engineer to CTO, along with how expectations change along the way. I use this book as a reference, and whenever I feel I am ready to take on a bigger challenge I re-read the chapter for the next level up.

Resilient Management — Lara Hogan

Lara is a management coach and founder of Wherewithall. Management can sometimes be more art than science, but Lara shares practical tactics that can be applied in the fuzziest of situations. I constantly refer to her Feedback Equation and Manager Voltron practices.

Difficult Conversations — Bruce Patton, et al

My first task as a new manager was to set clear expectations, and to provide feedback to engineers who weren’t meeting them. Delivering actionable, critical feedback that lands is a skill, and learning how to approach difficult conversations from a place of curiousity and collaboration has been invaluable to me in both my professional and personal life.

Becoming a Manager — Linda Hill

This book is not specific to software engineering management, and I was surprised to learn that the challenges you face when moving from IC to people manager are universal to a lot of professional disciplines. In particular, the myths vs reality

	Myth	Reality
Operative principle	Authority	Interdependence
Key players	Direct reports	Include those outside your formal authority
Source of power	Formal authority	Everything but formal authority
Focus	Managing 1:1	Managing one-on-one and leading the team
Desired Outcome	Control through compliance	Commitment through empowerment
Key challenges	Cope with complexity	Cope with complexity and change
Essential competencies	Technical	Technical, human, conceptual

Effective Delegation of Authority — Hassan Ossman

Scaling yourself is one of the bigger challenges of management, and it is easy to get into the habit of thinking “it will take me the same amount of time to explain as it will be to delegate it.” This may be true for simple tasks that sort well in the Eisenhower Decision Matrix:

…but you should delegate responsibilities, not tasks, and as with all things — the clearer the expectations are up-front, the better the outcome. This book is a quick read, but it made an outsized impact on my ability to scale — it also made me a leader who can grow the next generation of technical leaders.

Other Resources

So, You Want to Go to Code School!

Sat, 17 Apr 2021 00:00:00 GMT

I made a career change decision in 2013. I had been working as a lab technician at a biotech startup doing cancer and brain disorder research for 7 years, and had taken a certificate at nightschool to learn to program. Getting that first job in software development was hard, even with a certificate under my belt, so I decided to go to code school to focus on learning a web development framework. Code schools were relatively new in 2013, and what the industry lacked in organization it more than made up for in enthusiasm.

Code schools solve all kinds of challenges for the tech industry: not all problems in software require a CS degree; every industry requires software to function, driving unprecedented demand for programming skills; and career changers like myself can bypass trapdoors, which increases diversity. Getting accepted to a code school program gains you entry to a whole new world, but it can be hard to navigate without a guide.

Two years after graduating the program and working in the industry, I wrote the guide I would have liked to have had. “So, You Want to Go to Code School” includes everything I learned from the moment I decided to make the change through my first two years as a software developer. While I wrote this guide in 2015, and code schools have evolved considerably, I think there is still some value to be gleaned for early software developer careers.

One note before you dive in — the url for the book (codeschoolbook.com) is no longer mine 😀. I wish the new owners luck with their kickstarter campaign! Also, I am open to feedback — checkout my contact form or drop me an email.

Good luck on your journey!

So, You Want to Go to Code School! Download

Success With Snippets

Mon, 05 Apr 2021 00:00:00 GMT

Many engineers use code snippet managers to quickly auto-complete repetitious and predictable keystrokes. I have used Dash for many years to autocomplete test cases, import React from 'react' and to correct things I always misspell (in case you ever see it, my full name is Katherine not Kathering — why do I do this?).

Having learned a lot of strategies for effective communication, I started building snippets to keep them top of mind. These snippets help me to focus my thoughts, keep my audience in mind, and help me stop procrastinating. Whenever I had to create a presentation, write up a technical proposal, or to deliver constructive feedback.

Creating Presentations: Think of Your Audience First

I attended a workshop on “executive presence”. It was information dense, and I learned a tremendous amount about how to optimize your message for your audience. Much of the workshop centered on public speaking, so I also learned how to connect with your audience during a presentation, how to manage Q&A, and how to build and deliver a slide deck that enhances your message rather than distracts from it. Knowing that I would quickly forget many of these strategies, I distilled them into a snippet. I type `start, and my blank document is replaced with this template:

--------------------------------------
Goals
Audience: name your stakeholders and their currency
3 key takeaways
Call to action
--------------------------------------
Tell them what you will tell them.
Tell them.
Tell them what you told them.
--------------------------------------
Evoke emotion
Cut to the point
Arrange message for memorability
--------------------------------------
1. Agreement: provides a baseline and direction; big picture
2. Context: What is the problem and what is the pain? Brings the big picture closer
3. Story: narrative demonstrating a change from pain to no-pain
4. Connections: use analogies to explain and metaphors to highlight
5. Conclusion: summarize what was learned, and make your call to action.
--------------------------------------
Visuals should accentuate, reinforce, and repeat your message
--------------------------------------
Don't answer the question "what is X?",
  answer the question "why should I care about X?"
Going from less understanding to more understanding,
  don't be afraid to cover the basics:
  it accounts for everyone and validates the experts.
---------------------------------------

This snippet ensures that I start with the goal in mind, that I aim my content at the right audience, and that my takeaways are clear.

Writing Technical Proposals: Exploring Multiple Options

One of the common asks for a technical lead on a software team is to provide technical estimates and options for upcoming projects. These requests happen so frequently, I came up with a formula that made my communication clear and my results consistent. I type `tech and my blank document is replaced by this template:

# Technical Problem Writeup Template

## Problem

High level overview of the problem.
Determine your audience.
Summarize the problem at the right level of detail --
  * Executives: describe the WHY in terms of risks to the
      roadmap, team, reliability.
      Do not include technical detail.
  * PMs: describe the above but with some technical detail
  * Engineers: describe the technical challenges

## Options

* Include at least three options, one of which is to do nothing.
* List Pros and Cons of each option and be specific:
  “link from X to Y requires an extra query which will
  impact performance” rather than “bad customer experience.”
* Make a recomendation
* Include a level of effort
* Considered and Rejected
  * If any options were rejected, itemize and bullet
    point the reasons they were rejected

## Resources

Links that didn’t fit into the Details section,
  including other documents that describe or
  reference the problem at hand.

Using this snippet has leveled up my ability to communicate technical trade-offs to the right audience at the right level. I have been using it for almost two years, and I believe this strategy is one of the main accelerators of my career from senior enginer to architect.

Delivering Constructive Feedback: Stick to the Facts and Stay Curious

Delivering constructive feedback is hard, and there are more ways to do it wrong than to do it right, which is probably why we avoid it. After reading Resilient Management by Lara Hogan, I learned some incredible strategies for how to construct feedback that emphasizes empathy and improves the odds that your feedback will land as you intend. It doesn’t nearly do justice to Hogan’s beautiful blog, but when I need to carefully consider the impact of the feedback I need to deliver, I type `feedback and my blank document is replaced by this template:

-------------------------
## Feedback equation
Facts + impact + open question = good feedback

Keep asking "what is the impact of THAT" until you have the real impact.

## Frame the impact in terms of what THEY care about:

* Belonging? It makes you seem unapproachable...
* Improvement/progress? It slows down the project...
* Choice? It reduces our options...
* Equality/fairness? Others have to pick up the slack...
* Predictability? It disrupts the routine...
* Significance/status? It makes us lose trust...
-------------------------
## Open questions

* What are you optimizing for?
* What are you hoping to accomplish?

This snippet ensures that I am framing my constructive feedback in a way that will be most effective for the person receiving it. I consider their core needs, stick to the facts, and remain open to learning more about how they perceive things.

As a brief aside, I recently read “Difficult Conversations” by Douglas Stone, Bruce Patton, and Sheila Heen. They say very wisely that if you think you have all the information about a situation, then you are not ready to have a difficult conversation. This snippet is a good place to stick a reminder to get and stay curious about where someone else is coming from: I think I will add it so that I will remember it for next time.

Snippet managers are useful for more than just code completion

I use Dash, but there are many different code snippet managers out there. Find one that works for you and you will bring your written communication to the next level.

In case you are curious, here is the result of the snippet template I used to write this blog post:

--------------------------------------
Goal: To persuade the reader that snippets are useful outside of the code editor.

Audience: Software Engineers and Engineering Managers

3 key takeaways:

* use snippets to remind yourself what questions to ask
* use snippets to craft and focus your communication
* use snippets to break through writers block through prompts and structure

Call to action:

Use Dash or some other snippet manager to remind you of strategies you tend to forget.
--------------------------------------
...

Don't answer the question "what is X?", answer the question "why should I care about X?"

Why should they care about using snippets?
Snippets are a means to an end: providing consistent, clear communication builds trust with your peers and managers.
Career growth beyond an individual contributor role requires strong communication skills.
Building trust with your organization is good for your career growth.
---------------------------------------

Multiple Content Sources in Gatsby

Sat, 24 Oct 2020 00:00:00 GMT

Tell Gatsby about your new content

Using the gatsby-source-filesystem plugin, identify your new collection of content and give it a reference name.

Add this to the plugins section of your gatsby-config.js:

  plugins: [
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/content/cheatsheets`,
        name: `cheatsheets`,
    },

Tell Gatsby to add some node meta-data

When Gatsby is creating nodes out of your markdown files, tell it to detect what collection it is a part of and add that information to the node metadata:

Add this to your gatsby-node.js onCreateNode method:

exports.onCreateNode = ({ node, actions, getNode }) => {
  const { createNodeField } = actions

  if (node.internal.type === `MarkdownRemark`) {
    const collection = getNode(node.parent).sourceInstanceName

    createNodeField({
      name: 'collection',
      node,
      value: collection
    })
...
  }
}

With those two pieces in place, you can group your content by the collection they appear in:

If you are using the gatsby blog template like I am, you can fix some of those console errors by adding the new collection field to your schema:

In your gatsby-node.js file:

exports.createSchemaCustomization = ({ actions }) => {
  const { createTypes } = actions

  createTypes(`
...
    type MarkdownRemark implements Node {
      frontmatter: Frontmatter
      fields: Fields
    }
...
    type Fields {
      slug: String
      collection: String
    }
  `)
}

Update createPages to generate your collections

Start with a config object that describes each collection:

const COLLECTIONS = [
  {
    name: "blog",
    hasDetailPage: true,
    needsPagination: true,
    indexComponent: path.resolve("./src/templates/blog-list.js"),
    detailComponent: path.resolve(`./src/templates/blog-post.js`),
  },
  {
    name: "cheatsheets",
    hasDetailPage: true,
    indexComponent: path.resolve("./src/templates/cheatsheet-list.js"),
    detailComponent: path.resolve(`./src/templates/blog-post.js`),
  },
]

In order to build your collection pages, you need a generic function that will work for each of your collections. In my case, I wanted a collection of blog posts (a paginated index page and post page for each blog), and a collection of cheatsheets (a non-paginated index page and post page for each cheatsheet).

const buildCollectionPages = ({
  nodes,
  createPage,
  name,
  needsPagination,
  hasDetailPage,
  indexComponent,
  detailComponent,
}) => {
  const posts = filterNodes(name)(nodes)

  if (needsPagination) {
    // Paginated index pages
    const numPages = Math.ceil(posts.length / MAX_POSTS_PER_PAGE)
    Array.from({ length: numPages }).forEach((_, i) => {
      const currentPage = i + 1
      const previous = i === numPages - 1 ? null : currentPage + 1
      const next = i === 0 ? null : i

      createPage({
        path: i === 0 ? `/${name}` : `/${name}/page_${currentPage}`,
        component: indexComponent,
        context: {
          limit: MAX_POSTS_PER_PAGE,
          skip: i * MAX_POSTS_PER_PAGE,
          numPages,
          currentPage,
          next,
          previous,
        },
      })
    })
  } else {
    // Single index page
    createPage({
      path: `/${name}`,
      component: indexComponent,
    })
  }

  // Generate detail pages for each post in the collection
  if (hasDetailPage) {
    posts.forEach((post, index) => {
      const previous = index === posts.length - 1 ? null : posts[index + 1]
      const next = index === 0 ? null : posts[index - 1]
      const { slug } = post.fields

      createPage({
        component: detailComponent,
        path: slug,
        context: {
          slug,
          previous,
          next,
        },
      })
    })
  }
}

Query for all your data, ensuring that you fetch the collection field as well. Loop over your COLLECTIONS, and build the pages for each one by passing in the retrieved data:

const createCollectionPages = async function ({ graphql, actions, reporter }) {
  const { createPage } = actions

  const result = await graphql(`
    query {
      postData: allMarkdownRemark(
        sort: { fields: [frontmatter___date], order: DESC }
      ) {
        nodes {
          fields {
            collection
            slug
          }
          frontmatter {
            title
          }
        }
      }
    }
  `)

  COLLECTIONS.forEach(collection => {
    buildCollectionPages({
      ...collection,
      nodes: result.data.postData.nodes,
      createPage,
    })
  })
}

module.exports = createCollectionPages

The { graphql, actions, reporter } object is passed in to the createPages API from Gatsby. We call createCollectionPages in an async context in gatsby-node.js createPages

const createCollectionPages = require('./gatsby-helpers/create-collection-pages')

exports.createPages = async ({ graphql, actions, reporter }) => {
  await createCollectionPages({ graphql, actions, reporter })
}

If you aren’t generating any other types of pages, you can make it even more succinct:

exports.createPages = createCollectionPages

Awesome.

Styling Active Links in Gatsby

Sat, 24 Oct 2020 00:00:00 GMT

When Gatsby generates static pages, it helpfully adds a tag to the active nav item:

This makes it easy to add a style to the active nav so that your users always know what page they are on. Since I am using styled components, here is how I am styling the nav:

const StyledLink = styled(Link)`
  padding: 0.5rem;
  text-decoration: none;
  border: solid transparent;
  border-width: 0 0 thick 10px;

  &:hover,
  &[aria-current="page"] {
    color: var(--purple);
    border: solid var(--purple);
    border-width: 0 0 thick 10px;
    border-color: transparent var(--purple) var(--purple) transparent;
    border-radius: 0 0 20px 0;
  }
`

The hover styles now persist when someone clicks on a link!

Awesome.

Active Associations

Thu, 22 Oct 2020 00:00:00 GMT

Working in a legacy code base, you will come across a lot of join tables and belongs-to-through complexity. You can visualize these associations using an ERD, or you can find them in the console by using

<ModelName>.reflect_on_all_associations.map(&:name)

For example, I am testing out the Thredded forum gem, and found a lot of complex join tables to navigate. I wanted to find all the ways I could access data in the UserTopicFollows join table:

> Thredded::UserTopicFollow.reflect_on_all_associations.map(&:name)
=> [:user, :topic]

So, I can get there through a Topic. That makes sense, since it is a join table.

> Thredded::Topic.reflect_on_all_associations.map(&:name)
=> [:last_user,
 :slugs,
 :user,
 :messageboard,
 :user_detail,
 :posts,
 :first_post,
 :last_post,
 :topic_categories,
 :categories,
 :user_read_states,
 :user_follows,
 :followers,
 :likes]

And I can get to Posts from Topics. That also makes sense, since Posts belong to Topics.

2.7.0 :001 > Thredded::Post.reflect_on_all_associations.map(&:name)
 => [:user, :messageboard, :postable, :user_detail, :moderation_records, :user_notifications, :last_moderation_record]

It doesn’t look like I can get back to Topic from Post, however. Since a Post belongs to a Topic, I expected that association to be in the list. That :postable association was unexpected as well, and I couldn’t find a Postable model. It turns out that postable is an alias for Topic in a UserTopicFollow join table:

module Thredded
  class UserTopicFollow < ActiveRecord::Base
  ...
  alias_attribute :postable, :topic
  ...

While my head is still spinning a bit.. I found what I needed to get unblocked. 🙃

Using Styled Components With Gatsby

Sat, 10 Oct 2020 00:00:00 GMT

The styles assigned with styled components are applied at javascript runtime. They generate a css stylesheets specific to your component and inject it into the head. When you use styled components for rendering things like a nav or layout, there is a strong possibility that your content will arrive on the page before the CSS required to style it, causing a usually brief but very disconcerting flash of unstyled content:

This can be solved by ensuring that Gatsby inserts your important styles inline with your html when it generates the page.

Step one: ensure that gatsby knows you are using styled components

Follow the instructions on the Gatsby styled-components page:

$ npm i gatsby-plugin-styled-components

And import the plugin to your gatsby-config:

module.exports = {
  plugins: [
    ...
    `gatsby-plugin-styled-components`
    ...
    ],
}

Step two: add important styles to a GlobalStyle component

Inside your styles folder, create a GlobalStyles.js file and add your important styles like so:

import { createGlobalStyle } from "styled-components"

const GlobalStyles = createGlobalStyle`
...
  nav.custom-nav {
    margin-bottom: 3rem;

    ul {
      margin: 0;
      padding: 0;
      text-align: center;
      list-style: none;

      display: grid;
      grid-template-columns: auto 1fr 1fr 1fr;
      grid-gap: 2rem;
      align-items: center;
    }
  }
...
`

Step three: Inline your GlobalStyles in your Layout component

Ensure that your most important styles will be inlined on every page:

...
import GlobalStyles from '../styles/GlobalStyles'

export default function Layout = () => {

  return (
    <div>
      <GlobalStyles />
      <div className="site">
        <nav className="custom-nav">
          <ul>
            ...
          </ul>
        </nav>
        ...
      </div>
    </div>
  )
}

Swoosh underline on hover (CSS)

Sun, 04 Oct 2020 00:00:00 GMT

As I mentioned in my last post, I’m finally working on reskinning my personal blog and using Gatsby to build the site. I am also taking Wes Bos’s Master Gatsby course and realized that menus don’t need to be as static and boring as they historically have been. Since you can do almost anything with CSS, I decided to attempt a swoosh underline on hover, and it was more challenging than I expected.

Basic layout of the Nav is pretty dry markup:

<ul>
  <li>
    <Link to="/about">about</Link>
  </li>
  <li>
    <Link to="/projects">projects</Link>
  </li>
  <li>
    <Link to="/contact">contact</Link>
  </li>
</ul>

Using styled-components, I could experiment with different colors and treatments on the fly. You can see the result on the page above, but here is how I landed on the swoosh:

const StyledNav = styled.nav`
  display: flex;
  justify-content: flex-end;

  ul {
    list-style-type: none;
    display: flex;

    li {
      margin-right: 1rem;
      font-size: 1.25rem;

      a {
        padding: 0.5rem;
        text-decoration: none;
        border: solid transparent;
        border-width: 0 0 thick 10px;
      }

      a:hover {
        color: purple;
        border: solid purple;
        border-width: 0 0 thick 10px;
        border-color: transparent purple purple transparent;
        border-radius: 0 0 20px 0;
      }
    }
  }
`

Wrap the dry unordered list with the component, and voila! The key to not having janky hover styles is to ensure that your padding, margin, and borders are consistent between hover and non-hover states. Using transparent as the color helps with the borders width.

Opaque background images

Sat, 03 Oct 2020 00:00:00 GMT

I’m finally working on reskinning my personal blog and really enjoying the Gatsby ecosytem. I wanted to add a little color to the boilerplate, and to demonstrate to all observers that my stylistic bent is solidly stuck in the 90s.

Poking at the Gastby way of loading content, such as the avatar, it seems that the preferred way is to load the asset using graphql:

query BioQuery {
  avatar: file(absolutePath: { regex: "/profile-pic.jpg/" }) {
    childImageSharp {
      fixed(width: 50, height: 50, quality: 95) {
        ...GatsbyImageSharpFixed
      }
    }
  }
  ...

It turns out that there is a plugin for everything in Gatsby, including one for adding background images using react. Any front-end developer worth their salt would read those docs, get super excited about processing background images at runtime. I am not that developer. I used css.

body::after {
  content: "";
  background: url(../content/assets/splatter-background.png);
  background-size: contain;
  opacity: 0.2;
  top: 0;
  left: 0;
  bottom: 0;
  right: 0;
  position: absolute;
}

Recursive Rendering and Event Bubbling

Sun, 01 Apr 2018 00:00:00 GMT

I have been working with dense tree structures recently, and rendering them using React. Trees lend themselves well to recursion, so I set about making a component that could render itself and all subsequent children with ease. I thought I was being mightily clever, until I tried to add some behaviour to my recursive component. Things were not behaving as they should, and I was stumped for quite some time. When I finally stepped back from the trees to see the forest (pun intended), I had to step all the way back to HTML-101 to find the answer.

Here is my clever component:

export default class ListItem extends PureComponent {
  renderChildren = arr => {
    return <ul>{arr.map(this.renderChild)}</ul>;
  };

  renderChild = item => <ListItem {...item} />;

  render() {
    const { name, children } = this.props;

    return (
      <li>
        {name}
        {children && this.renderChildren(children)}
      </li>
    );
  }
}

When called from the parent container with the root of the tree, this component will render all nodes of the tree as unordered lists of items:

Of course, with very deep trees, it is useful to clear away visual noise by collapsing parts of the tree, so I added an onClick handler to each list item with children:

export default class ListItem extends PureComponent {
  state = {
    showChildren: true
  };

  onClick = e => {
    this.setState({ showChildren: !this.state.showChildren });
  };

...

  render() {
    const { name, children } = this.props;
    const { showChildren } = this.state;

    return (
      <li onClick={this.onClick}>
        {name}
        {children && showChildren && this.renderChildren(children)}
      </li>
    );
  }
}

Setting a showChildren state on the ListItem instance will hide parts of the list by simply not rendering them, rather than hiding them using traditional CSS. You can imagine my chagrin, then, when I clicked on an item, and the entire tree collapsed into the root:

Troubleshooting recursion is tricky. Was the state change leaking from one instance of ListItem to another? They should be independent instances, but what if they shared a state reference?

Finally, I turned my suspicions on the click handler.

I left the React docs behind and went back to the primer on how the DOM handles events. This is where I found the answer to the overzealous-click problem — it wasn’t ocurring because of using recursion, it was happening because of event bubbling (:face-palm:).

[When a user clicks on a ListItem, t]he browser checks to see if the element that was actually clicked on has an onclick event handler registered on it in the bubbling phase, and runs it if so… Then it moves on to the next immediate ancestor element and does the same thing, then the next one, and so on until it reaches the <html> element.

Because there was a click handler registered on every list item, a click on any one of them would be handled by every one of them. Fortunately, Javascript-Events-101 also prescribes a solution: e.stopPropagation(), which stops the event from bubbling up the ancestry tree.

My final onClick handler:

onClick = e => {
  e.stopPropagation();

  const { showChildren } = this.state;
  this.setState({ showChildren: !showChildren });
};

And the nested result I wanted:

Styled Components and Storybooks

Fri, 02 Mar 2018 00:00:00 GMT

I recently discovered two new react development tools that have changed how I approach React component development: Storybook and styled-components. I can quickly prototype a component with its data requirements, styles, and behaviour in one file, and in isolation from the rest of my application. This rapid prototyping has greatly increased my development cycle time, as I spend more time in code and less time flipping between the browser window, the browser console, css files, jsx files, and my terminal.

Using these libraries has also had the pleasant side-effect of helping me design more pure components: instead of using whatever context is available in my giant application, I think about my component interface with real encapsulation in mind.

Storybook

Storybook creates a hot-reloading development sandbox outside of your running application, where you can create components free from global app dependencies. Building a component outside of your application context makes you think about what props are really needed to render a component. Is one component actually two components? What is the least amount of information your component needs to be a single atom.

Writing a story for storybook is like writing a spec: you describe the features of the component variant (“with long title”), and provide a callback that renders your component with props that you specify.

storiesOf("Playbill", module)
  .add("basic", () => (
    <Playbill
      onClick={action("navigate to playbill")}
      title="foo"
      imageUrl="https://virtualplaybill.s3.amazonaws.com/1517893742975_Magellanica"
    />
  ))
  .add("with long title", () => (
    <Playbill
      onClick={action("navigate to playbill")}
      title="we are proud to present a presentation"
      imageUrl="https://virtualplaybill.s3.amazonaws.com/1517893742975_Magellanica"
    />
  ));

You can interact with your components and see the consequences of those interactions in the action pane below.

Styled-components

Styled-components remove the disconnect between styles and components (they are literally connected with a hyphen 😄). Before styled-components, there were clear sides to the Great CSS debate: why do we even need CSS if we inline component styles? Is it just for performance? Why not use the React render cycle to handle dynamic styles?

I am coming pretty late to the party here, but as a solid supporter of CSS over inline styles, styled components have completely won me over. Instead of filling your markup with style={{}} attributes, you create a new component that expresses the styles you specify.

import styled from "styled-components";

const PlaybillContainer = styled.div`
  height: 415px;
  width: 250px;
  border: 1px solid black;
  display: flex;
  flex-direction: column;
  justify-content: space-between;
`;

const Playbill = () => {
  return <PlaybillContainer />;
};

Under the hood, styled-components will generate a stylesheet and inject it into the head of your page:

<style type="text/css" data-styled-components="fAjaVQ cjXYkN ktobWA bNUGoX fSQmFU sPlKa jPyvbr cDsQtG ddUpJT buzCko llbHNh wUSUC eapGFo gcivQF llqivM cHJMlW" data-styled-components-is-local="true" nonce="undefined">
...
/* sc-component-id: sc-fjdhpX */
.sc-fjdhpX {}.cHJMlW{display:block;height:300px;width:250px;}</style>

Styled-components have effectively put an end to the styles-in-Javascript debate. They are an expressive way to have the best of both worlds: inline styles and cascading style sheets.

Quick start with both Storybook and styled-components

# install create-react-app
# for babel boilerplate
> npm i -g create-react-app
> create-react-app storybook-test-app
> cd storybook-test-app
> npm i

# install storybook and styled-components
> npm i -g @storybook/cli
> getstorybook
> npm i styled-components
> npm run storybook

# open your new storybook at
# http://localhost:9009/

Your stories are saved in src/stories, and you can import them from an adjacent components folder for both quick development, and quick incorporation into your application.

Example

I am doing a React rebuild of my side project Virtual-Playbill, and storybook is the perfect place to play around with tweaks to the UI.

My component

import React from "react";
import styled from "styled-components";

const PlaybillContainer = styled.div`
  height: 415px;
  width: 250px;
  border: 1px solid black;
  display: flex;
  flex-direction: column;
  justify-content: space-between;
`;

const PlaybillHeader = styled.div`
  padding: 10px 0 10px 0;
  font-size: x-large;
  max-height: 2em;
  text-align: center;
  width: 100%;
  background-color: yellow;
  border-bottom: 1.5px solid black;
  text-transform: uppercase;
  font-weight: bold;
  letter-spacing: 2.5px;
`;

const PlaybillTitle = styled.div`
  text-align: center;
  font-weight: bold;
  font-size: large;
  max-width: 95%;
`;

const PlaybillImage = styled.img`
  display: block;
  height: 300px;
  width: 250px;
`;

const Playbill = ({ title, imageUrl, onClick }) => {
  return (
    <PlaybillContainer onClick={onClick}>
      <PlaybillHeader>Virtual Playbill</PlaybillHeader>
      <PlaybillTitle>{title}</PlaybillTitle>
      <PlaybillImage src={imageUrl} />
    </PlaybillContainer>
  );
};

export default Playbill;

My stories

import React from "react";

import { storiesOf } from "@storybook/react";
import { action } from "@storybook/addon-actions";

import Playbill from "../components/playbill";

storiesOf("Playbill", module)
  .add("basic", () => (
    <Playbill
      onClick={action("navigate to playbill")}
      title="foo"
      imageUrl="https://virtualplaybill.s3.amazonaws.com/1517893742975_Magellanica"
    />
  ))
  .add("with long title", () => (
    <Playbill
      onClick={action("navigate to playbill")}
      title="we are proud to present a presentation"
      imageUrl="https://virtualplaybill.s3.amazonaws.com/1517893742975_Magellanica"
    />
  ));

Between styled-components and storybook, I don’t need to leave my component file until the component is finished. Feedback cycles are tight, which means I can try out different variations quickly before dropping the component into my application.

Awesome! 😎

decorate your javascript

Wed, 28 Feb 2018 00:00:00 GMT

Javascript decorators are a form of metaprogramming: they add functionality to classes and properties. Unlike the GoF pattern, where decorators modify instances of a class, Javascript decorators are run when the class, method, or property is installed, modifying all instances.

Decorators are useful for adding extra functionality to behaviours and properties that would otherwise look like boilerplate — such as cacheing, access control, logging, or instrumentation.

How does it work?

A decorator function runs before the object it decorates is installed on the prototype. When you define an undecorated object like this:

class ExampleWithoutDecoration {
  doWork() {
    console.log('can\'t you see I\'m working here?')
  }
}

The Javascript engine creates an object and installs the doWork method on its prototype:

Object.defineProperty(ExampleWithoutDecoration.prototype, 'doWork', {
  value: specifiedFunction,
  enumerable: false,
  configurable: true,
  writable: true
})

When you define a DECORATED method like this:

class ExampleWithDecoration {

  @DecoratingIsFun
  doWork() {
    console.log('can\'t you see I\'m working here?')
  }
}

The Javascript engine saves some temporary state, runs the decorator function, and then installs the doWork method on the object’s prototype.

let methodDescription = {
  type: 'method',
  initializer: () => specifiedFunction,
  enumerable: false,
  configurable: true,
  writable: true
};

methodDescription = DecoratingIsFun(ExampleWithDecoration.prototype, 'doWork', methodDescription) || methodDescription

defineDecoratedProperty(ExampleWithDecoration.prototype, 'doWork', methodDescription);

function defineDecoratedProperty(target, { initializer, enumerable, configurable, writable }) {
  Object.defineProperty(target, { value: initializer(), enumerable, configurable, writable })
}

In this case, the DecoratingIsFun method is run with this set to the object prototype, and it has the opportunity to modify/return a methodDescription, or use the previously specified methodDescription.

Consider DecoratingMakesSense, which makes the doWork method non-writeable:

const DecoratingMakesSense = (object, methodName, description) => {
  console.log('Decorating makes sense')

  description.writable = false
  return description
}

class ExampleWithDetailedDecoration {
  @DecoratingMakesSense
  doWork() {
    console.log('can\'t you see I\'m working here?')
  }
}

const makesSense = new ExampleWithDetailedDecoration()
makesSense.doWork()
makesSense.doWork = () => console.log('some other function')

> node build/main.js
Decorating makes sense
can't you see I'm working here?

/decorator-example/build/main.js:164
makesSense.doWork = function () {
                  ^

TypeError: Cannot assign to read only property 'doWork' of object '#<ExampleWithDetailedDecoration>'
    at Object.defineProperty.value (/decorator-example/build/main.js:164:19)
...

Getting started

Since decorators are currently in the proposal stage, getting started requires a little tweaking of your standard babel/webpack/linter configs.

Babel

> npm install -g webpack
> npm install --save-dev babel-core babel-loader babel-preset-es2015 babel-plugin-transform-decorators-legacy

In your .babelrc

{
  "presets": [
    "es2015"
  ],
  "plugins": [
    "transform-decorators-legacy"
  ]
}

Webpack

Here is a barebones webpack.config.js example

var path = require('path')

module.exports = {
  entry: './index.js',
  output: {
    path: path.resolve(__dirname, 'build'),
    filename: 'main.js'
  },
  module: {
      loaders: [
    {
      test: /\.js$/,
      exclude: /node_modules/,
      loader: 'babel-loader',
      query: {
        cacheDirectory: true,
        plugins: [
          'transform-decorators-legacy',
        ],
        presets: ['es2015'],
      },
    }
  ]
  },
  stats: {
    colors: true
  }
}

(The important bit is to add ‘transform-decorators-legacy’ to the plugins array.)

Linting error — or is it? 🤔

If you use VSCode, you will probably run across a linter error that says

[js] Experimental support for decorators is a feature that is subject to
change in a future release. Set the 'experimentalDecorators' option to
remove this warning.

This is an error in the VSCode JS support, rather than a linter error.

Add a jsconfig.json file to your project root with the following contents:

{
  "compilerOptions": {
    "experimentalDecorators": true
  }
}

Be sure to restart VSCode, and the problem should go away. If it doesn’t, follow this thread for further information.

Build and run

> webpack
Hash: 63cf378bd6d165758ed8
Version: webpack 3.8.1
Time: 457ms
  Asset     Size  Chunks             Chunk Names
main.js  4.47 kB       0  [emitted]  main
   [0] ./index.js 1.7 kB {0} [built]
   [1] ./decorator.js 195 bytes {0} [built]

> node build/main.js
Decorating is fun
can't you see I'm working here?

For a detailed example checkout this repo. For more examples and a deep dive, checkout wycats’s decorator spec

least recently used (LRU) cache

Fri, 23 Feb 2018 00:00:00 GMT

I was recently challenged to implement a least recently used (LRU) cache in javascript, which taxed both my object-oriented javascripting chops as well as my hazy memory of how to implement a linked list. An LRU cache discards least recently used items first (wikipedia). Not only do you have to limit the size of the list, but you have to track when an item has been accessed recently and ensure it is not removed before an item that has been accessed less recently.

Judging from the scores of implementations available from my first Google search, this appears to be a fairly common interview question. Here is the approach that I took:

Maintain performance by storing items in a hash map
Use a queue-type linked list to keep track of access order: a new item is added to the end of the queue, while an existing item is removed from its place in the queue and re-added to the end of the queue.
Keep the cache at the specified size by lopping off the head of the list (and removing those items from the map).
Do it all using object oriented design principles.

The API/object design

A simple cache should not have too many bells and whistles. It needs a get and a set, and possbily a size (useful for testing, probably not something most users desperately need). It should take a specific cacheSize as well, so we can set limits on the cache at startup.

const LRU = function (cacheSize) {
  this._maxSize = cacheSize

  this.get = (key) => {}
  this.set = (key, value) => {}
  this.size = () => {}
}

The data structures

To access stored values in the cache in the most performant way, we store them with a key in a hash map:

// ...
this._map = new Map()
// ...

In order to keep track of order access, we need to keep references to the head and the tail of the list, as well as an object to store the cached value and its place in the queue:

//...
this._head = null
this._tail = null
//...

const Node = function(key, value) {
  this.key = key;
  this.value = value;
  this.next = null;
  this.prev = null;
};

The implementaton

const LRU = function(cacheSize) {
  this._maxSize = cacheSize;
  this._map = new Map();
  this._head = null;
  this._tail = null;

  this.size = () => this._map.size;

  this.get = key => {
    const item = this._map.get(key);

    if (item) {
      this._dequeue(item);
      this._enqueue(item);
      return item.value;
    }
    return;
  };

  this.set = (key, value) => {
    const node = new Node(key, value);

    if (this._map.has(key)) {
      const existingNode = this._map.get(key);
      this._dequeue(existingNode);
    }

    this._enqueue(node);
    this._trimList();

    return value;
  };

  this._trimList = () => {
    if (this._map.size > this._maxSize) {
      this._dequeue(this._head);
    }
  };

  this._enqueue = node => {
    this._map.set(node.key, node);
    if (!this._head) {
      this._head = node;
      this._tail = node;
    } else {
      this._tail.next = node;
      node.prev = this._tail;
      node.next = null;
      this._tail = node;
    }
    return node.value;
  };

  this._dequeue = node => {
    if (this._head === node) {
      const { next, key } = this._head;
      if (next) {
        this._head = next;
        this._head.prev = null;
      } else {
        this._head = null;
        this._tail = null;
      }
    } else {
      const { prev, next } = node;
      prev.next = next;
      next.prev = prev;
    }
    this._map.delete(node.key);
  };

  this.toString = () => {
    const itr = this._map.entries();
    const entries = [];
    for (let item of itr) {
      const [key, node] = item;
      const { value } = node;
      entries.push([key, value]);
    }
    return entries;
  };
};

const Node = function(key, value) {
  this.key = key;
  this.value = value;
};

module.exports = LRU;

Checkout this repo for the full implementation and tests.

writing test cases in java with enums

Sat, 18 Mar 2017 00:00:00 GMT

I recently discovered some new (to me) features of enums, and so naturally I am now using them for everything. One of these new-to-me features is using a constructor to give enum members properties and behaviour.

For example, if I were making a list of colours, a simple enum would look like this:

enum Colors {
    RED,
    BLUE,
    YELLOW;
}

Everywhere in my code where I used to use the string "red", I could replace with a less-spelling-error-prone-and-IDE-friendly Colors.RED. If I want to iterate over all the colors in my collection, I can use the values method:

...
public void loopColorEnum() {
    for (Color color : Color.values()) {
        System.out.println(color);
    }
}
...

// console output:
// RED
// BLUE
// YELLOW

If I wanted to store the hex value for each colour, I can add a constructor and provide arguments for each member of the enum:

enum Color {
    RED("#FF0000"),
    BLUE("#0000FF"),
    YELLOW("#FFFF00");

    final String hex;
    Color(String hexCode) {
        this.hex = hexCode;
    }
}

Now I can access the hex property on all of my colours:

...
    public void loopColorEnum() {
        for (Color color : Color.values()) {
            System.out.println(color.hex);
        }
    }
...

// console output:
// #FF0000
// #0000FF
// #FFFF00

Finally, if I wanted to create a custom output for my enum that included the hex value, I can add a toString() function:

enum Color {
    RED("#FF0000"),
    BLUE("#0000FF"),
    YELLOW("#FFFF00");

    String hex;
    Color(String hexCode) {
        this.hex = hexCode;
    }

    public String toString() {
        return String.format("Color: %s (Hex: %s)", super.toString(), hex);
    }
}

...
    public void loopColorEnum() {
        for (Color color : Color.values()) {
            System.out.println(color);
        }
    }
...

// console output:
// Color: RED (Hex: #FF0000)
// Color: BLUE (Hex: #0000FF)
// Color: YELLOW (Hex: #FFFF00)

Pretty cool! Pair a constructor with the values() iterable, and you have yourself a handy-dandy test framework for repetitive but important tests. For example, I was re-visiting an old exercise from a data structures class and wanted to write a bunch of test cases without the repetitive boilerplate code for each one — so I built an enum:

public class CalculatorTest extends InfixCalculator {

    private enum Expression {
        ADDITION("1+2+3", 6),
        SUBTRACTION("9-4", 5),
        MULTIPLICATION("5*4*2", 40),
        DIVISION("9/3", 3),
        ADDITION_WITH_SPACES("5 + 7", 12),
        MULTI_DIGITS("12 + 3 - 5", 10);

        final String expression;
        final int result;
        Expression(String expression, int result) {
            this.expression = expression;
            this.result = result;
        }
    }

    @Test
    public void test_solves_an_expression() {
        Calculator calc = new Calculator();
        for (Expression exp : Expression.values()) {
            assertEquals("for expression " + exp.expression,
                    exp.result,
                    calc.calculate(exp.expression));
        }
    }
}

When I create a new test case, I just add another member to the enum, and provide the string input and the expected result. Reducing the boilerplate made developing around edge cases far more wieldy, and I spent more time writing the actual code of the Calculator than I did creating tests.

Adding a constructor, methods and properties to an enum makes it looks suspiciously like a class. In fact, that is how enums are implemented under the hood: they are classes with benefits, such as:

retrieve the constant value with valueOf,
guaranteed singletons,
clarity in switch statements, and
built in ordering of members — an enum can act like an array, and return the order of a member the ordinal() method

Sure, you could accomplish most of these using a class with public static final constants and a few hand-rolled methods, but you will impress your neighbours more with a well crafted enum.

Awesome.

iptables: middleware for network management

Sat, 25 Feb 2017 00:00:00 GMT

Some companies don’t allow their employees to use social media at work, and some countries don’t allow their citizens to access news outside of controlled outlets. How do these institutions control access to information that seems so available in other contexts? Through iptables.

When you enter facebook.com in the address bar of your browser (or open the native app on your phone), the information exchanged between your device and the servers at Facebook is broken up into tagged chunks of data called packets. Iptables contain chains of rules, and each rule contains a policy for how to handle the network packets sent to and from your computer.

For example, a company that wants to block incoming Facebook data can set up a custom rule that sends all packets from Facebook to the garbage instead of delivering them to the browser.

If a packet fails to match any of the policy rules in the chain, it is usually dropped by default.

Iptables are useful in many scenarios:

to build your own firewalls,
to restrict access to specific sites (like Facebook),
to load balance incoming web traffic,
to block a DoS attack, or
to simulate a network or service failure in order to test the resilience of your system (my team uses this strategy for some game-day activities)

Administrator privileges are required, but you can use iptables on linux or pfctl on mac to in order to inspect or change any of your iptable rules.

While I am neither a system administrator, nor particularly interested in bespoke network management, I do occasionally install software that will help me focus by blocking my access to distracting sites — it is always good to know some of the magic going on under the hood.

Resources

for Linux: control your network traffic
for Mac: how to use pfctl
an interesting primer on common iptable rules

thread safety

Sat, 04 Feb 2017 00:00:00 GMT

While wading into the wide world of high throughput production Java, I have been enjoying guidance from “Java Concurrency in Practice” by Brian Goetz, as well as my friend and coworker, David Copeland. In a recent talk, David boiled down the range of concurrency problems to three main issues: atomicity, visibility, and ordering conflicts.

Atomicity

An atomic operation is one that executes in a single operation. A variable assignment (int x = 2;) is an example of an atomic operation (mostly* — see footnote). Incrementing a variable, on the other hand, is NOT an atomic operation.

public class NonAtomicCounter {
  private int count = 0;

  public int getCount() { return count; }
  public void doWork() { count++; }
}

While count++ appears to be a single action it is actually three:

read count,
modify count, and
write count.

If ThreadA and ThreadB share a NonAtomicCounter and both call doWork(), ThreadA can read the value of count while ThreadB is modifying it, making count vulnerable to lost updates.

Be suspicious of any shared state that is involved in read-modify-write or test-then-act sequences of actions.

Visibility

Once a thread has read a value, there is no guarantee that it will ever check to see if another thread has modified that value.

public class Main {
    static boolean stop;

    public static void main(String[] args) throws InterruptedException {

        new Thread(() -> {
            while (! stop) {
                System.out.println("still working");
            }
        }).start();

        new Thread(() -> { stop = true; }).start();
    }
}

Each thread can cache a local copy of stop, which means they will read the value once and then never check to see if it has been changed.

Be suspicious of loops that are gated by a variable that is visible to other threads.

Ordering

The JVM has a just-in-time compiler that optimizes execution at runtime. The memory model guarantees deterministic behaviour within a thread, but makes no promises about how instructions are ordered in the meantime.

public class Main {
    static int A;
    static int B;

    public static void main(String[] args) throws InterruptedException {

        Thread t1 = new Thread(() -> {
            int localA = A;
            B = 1;
            System.out.println("local A: " + localA);
        });

        Thread t2 = new Thread(() -> {
            int localB = B;
            A = 2;
            System.out.println("local B: " + localB);
        });

        t1.start();
        t2.start();
    }
}

/* POSSIBLE LOCAL VALUES
* localA: 0, localB: 0
* -------------
* localA: 0, localB: 1
* --------------
* localA: 2, localB: 1 => THIS SHOULD BE IMPOSSIBLE, RIGHT?
* --------------
* localA: 2, localB: 0 => WAT?
*/

How is it possible that t1 sees that A = 2? From the docs:

“The semantics of the Java programming language allow compilers and microprocessors to perform optimizations that can interact with incorrectly synchronized code in ways that can produce behaviors that seem paradoxical.""

What this means, is that even though the instructions are written in such a way that localA should not be able to see the reassignment of A, the compiler can reorder instructions to optimize evaluation. As long as the result of the computation is deterministic within a thread, the compiler is free to change the execution order, inline assignments, simplify algebra, etc.

In the above example,

int localB = B;
A = 2;

is the same as

A = 2;
int localB = B;

When the instructions for thread1 and thread2 are interleaved, thread1 will occasionally see A = 2.

Be suspicious of operations involving multiple variables and ordering requirements.

Thread Safety Analysis Guidelines

If you are dealing with a thread safety issue, ask these three questions:

Do your threads share mutable state? Look for public getters and setters, or methods that return references to state rather than copies or primitive values.
Do your threads share multi-variable states (invariants)? Can one component change independently from others when they should always be treated as a single unit?
For any publicly mutable state, is it an atomicity, visibility, or ordering issue?

Solutions for thread safety issues

The Java language has a few solutions to offer:

The `volatile` keyword

Adding the keyword volatile to a variable declaration tells the JVM not to use a cached value for any read operations. This keyword will ensure that writes to a variable are always visible to other threads, but will not fix atomicity or ordering issues.

The `synchronized` keyword

Using a synchronized lock around a variable will address all three issues, at the cost of concurrency. If only one thread at a time can access your data, your program is single threaded. Common lock-gotchas include locking writes but not reads!

The `final` keyword

Making variables final addresses all three issues, but requires your state to be immutable. This approach is most useful in composition with other techniques.

Use guava libraries

Google has published a library of thread-safe collections that handle concurrent access for you. ConcurrentHashMap is one of my favourites, and you can check out the user guide for more.

Note

A variable’s declaration and assignment are really two operations: the allocation of memory to hold the value, and the writing of that value into memory. If a variable is declared and assigned within an object constructor, the object will not be published without variable initialization. However, if it is declared and assigned anywhere else, there are no such guarantees. For the purpose of this example, I am completely ignoring these implications :D

effectively final

Wed, 11 Jan 2017 00:00:00 GMT

I have been working on a Java concurrency bug at work for the last few weeks, and the intertwining concepts of immutability and publication have recurred several times. With the help of “Java Concurrency in Practice,” by Brian Goetz, I am starting to make sense of these concepts, and why these kinds of bugs can be difficult to reproduce.

What does it mean to publish an object?

“Publishing an object means making it available to code outside of its current scope, such as by storing a reference to it where other code can find it, returning it from a nonprivate method, or passing it to a method in another class.” (Goetz et al.)

An object that is published is available for use outside of its declared context. Publication can be direct or indirect. Storing a reference to an object in a public static field, directly publishes that object to any client of the class.

public class PublishedObject {

    public static void main(String[] args) {
        System.out.println(DirectlyPublishedObject.stringArray[0]);

        DirectlyPublishedObject.stringArray[0] = "changed";

        System.out.println(DirectlyPublishedObject.stringArray[0]);
    }

    public static class DirectlyPublishedObject {
        public static String[] stringArray = {"gotcha"};
    }
}

/*
* Console output:
* gotcha
* changed
*/

DirectlyPublishedObject.stringArray can be retrieved and modified by any client that has access to the class (in this case, PublishedObject#main is the client). An object can also be indirectly published if it is a visible member of another object that is published:

public class PublishedObject {

    public static void main(String[] args) {
        IndirectlyPublishedObject ipo = new IndirectlyPublishedObject();
        String[] myPersonalStringArray = ipo.getStringArray();

        System.out.println(myPersonalStringArray[0]);
        myPersonalStringArray[0] = "changed again";
        System.out.println(myPersonalStringArray[0]);
    }

    public static class IndirectlyPublishedObject {
        private String[] stringArray = {"gotcha"};

        public String[] getStringArray() {
            return stringArray;
        }
    }
}

/*
* Console output:
* gotcha
* changed again
*/

If ObjectA obtains a reference to ObjectB, it doesn’t matter if ObjectB is declared private, it has now been publicly published.

Why does publication matter? Shared objects have different publication requirements depending on whether they are mutable. From Goetz:

“Immutable objects can be published through any mechanism;
Effectively immutable objects must be safely published;
Mutable objects must be safely published, and must be either thread safe or guarded by a lock.”

What does it mean to for an object to be immutable?

An immutable object must satisfy these criteria:

The class is not extensible (no methods can be overridden)
All fields are marked final
All fields are assigned in the constructor
No internal state is modifiable by the client of the class (ie: no setters)
No fields that reference mutable objects are made available to clients of the class

public final class ImmutableObject {
    private final String name;

    public ImmutableObject(String name) {
        if (null == name)
            throw new IllegalArgumentException("Must supply a valid name.");

        this.name = name;
    }

    public String getName() {
        return name;
    }

    public static void main(String[] args) {
        ImmutableObject io = new ImmutableObject("keighty");
        System.out.println("name = " + io.getName());
    }
}

The ImmutableObject will not have any state other than a single field name that is initialized to the String value passed to its constructor. Because name is marked final and initialized in the constructor, there is no danger of encountering an instance of ImmutableObject in the wild whose name is not initialized.

If an object violates some of the requirements for immutability, but its state will never be modified after publication, it will still be treated as immutable by the compiler and JVM — it is effectively immutable.

The String class is “effectively immutable”

In Java, an instance of the String class is not technically immutable, even though Strings are constant and cannot be modified after creation. They have another property (a private hash) that is lazy loaded only as it is needed:

A String is considered effectively immutable because its internal state is completely encapsulated and its external state isn’t changed after it has been published. Even though the hash field is not final, and is only assigned as it is required (a performance optimization), it is not modifiable by any client. The program can treat Strings as if they are immutable, even if they don’t strictly fit all of the criteria.

How can I make my objects effectively immutable?

By ensuring that the internal state is completely encapsulated, and any mutable internal objects returned by a public method are copies, not references, you too can enjoy the benefits of at-will publication.

public class ImmutableObject {

  public static class EffectivelyImmutableObject {
    private String[] stringArray = {"gotcha"};

    public String[] getStringArray() {
      // return a COPY of the mutable object, not a reference
      return Arrays.copyOf(stringArray, stringArray.length);
    }
  }

  public static void main(String[] args) {
      EffectivelyImmutableObject eio = new EffectivelyImmutableObject();
      String[] myPersonalStringArray = eio.getStringArray();

      System.out.println(eio.getStringArray()[0]);

      myPersonalStringArray[0] = "changed a copy only";

      System.out.println(myPersonalStringArray[0]);
      System.out.println(eio.getStringArray()[0]);
  }
}
/*
* Console output:
* gotcha
* changed a copy only
* gotcha
*/

RESOURCES

Peierls, Tim; Goetz, Brian; Bloch, Joshua; Bowbeer, Joseph; Lea, Doug; Holmes, David (2006-05-09). Java Concurrency in Practice. Pearson Education. Kindle Edition.
javadoc for String

annotating java

Sun, 08 Jan 2017 00:00:00 GMT

Annotations are a means of adding metadata to Java source code. They can be applied to declarations of classes, fields, methods, and types (as of Java8) in order to pass instructions to the compiler. Annotations can even be evaluated at Runtime. For example, @Inject is an annotation from the Dagger dependency injection library used to indicate which constructor should be used to instantiate a class. @Inject also tells Dagger to create the dependencies identified in the constructor parameter list.

Provided by `java.lang`

@Deprecated: don’t use the marked object. This annotation should be accompanied by a comment explaining the reason and alternative.
@Override: this method overrides a superclass method.
@SuppressWarnings: the compiler will not report warnings generated by this method.

Create your own

Annotations can be used to replace verbose code comment templates, and since they are ostensibly interfaces, they can be created the same way you would any other interface.

// src/CustomAnnotation.java
import java.lang.annotation.Documented;

@Documented // Annotation of an Annotation -- so meta! [1]
@interface CustomAnnotation { // [2]
    int id(); // [3]
    String description();
    String author() default "[unknown]"; // [4]
}

CustomAnnotation has an annotation of its own! The @Documented annotation ensures that CustomAnnotation and its properties will be included in the Javadoc when it is generated.
Declare a new annotation using @interface instead of interface
Fields id and description are considered required because they are not given default values. Values for id and description must be provided when using @CustomAnnotation.
author is an unrequired field because a default value will be used if one is not provided.

Annotations for Annotations

When defining an annotation, you can indicate how it should be used:

@RetentionPolicy: How should the annotation be stored? Only in source, included in the generated Class, or passed all the way to the JVM for Runtime evaluation?
@Documented: Include the annotation when generating Javadocs (default is false).
@Target: A list of locations where the annotation can be used (constructor, field, method, etc).
@Inherited: The annotation properties can be inherited from the superclass.
@Repeatable: The annotation can be used more than once (declaring multiple authors, for example).

Using your CustomAnnotation

Use @CustomAnnotation to decorate your code:

// src/Main.java
public class Main {

    ...

    @CustomAnnotation(id = 1, description = "Custom description")
    public void printSlogan() {
        System.out.println("This method uses a @CustomAnnotation");
    }

    @CustomAnnotation(id = 2, description = "Custom banner", author = "keighty")
    public void printBanner() {
        System.out.println("This is a printed banner for: " + message);
    }
}

When you generate the javadoc for Main.class, the annotations will be included in the method descriptions:

$  javadoc src/com/keighty/Main.java -d .

Annotating a field vs annotating a parameter

There are no hard rules about whether one should annotate a field or a constructor parameter, but which ever you choose you should use that pattern consistently.

If you use the @Nonnull annotation in the constructor, the compiler will warn if there is any use of the constructor where the parameter could be null. If you use the @Nonnull annotation on the field, you will have to do your own null checking when assigning the field.

RESOURCES

Keighty's 2016 - by the numbers

Tue, 27 Dec 2016 00:00:00 GMT

Many people would like a do-over of 2016, but this was a pretty epic year for me. I was traveling pretty constantly, reading voraciously, and enjoying each left-turn my career has taken me. I started the year helping to build instrumentation for single-page applications, bounced into some UI projects using React, and spent the last quarter ensconced in a crash-course in Java Concurrency. Other highlights include achieving US Residency (so long for now, Canada 🇨🇦 ), seeing a more reasonable amount of theater, and shipping more diverse side-project code than ever. Check out my personal infographic (thanks canva.com!):

browser-sync is the easiest webserver EVER

Sun, 23 Oct 2016 00:00:00 GMT

Browser-sync is the fastest way to spin up a local web server. It will even open your default browser with the entry point you specify, and live-reload your changes as you make them. It couldn’t possibly be any simpler to test out a site you’re developing locally.

1. Install browser-sync

To get started, install the browser-sync package from npm:

$ npm install -g browser-sync

2. Configure the server

Setup a simple static project folder like this (gist):

.
├── bs-config.js
├── foo
│   └── index.html
└── index.html

In your bs-config.js file, list the configuration options for how browser-sync should serve your files, including which file extensions it should watch for changes ("*html","*js","*css"), and what resources should be served by a new route name ("/foo": "foo").

// bs-config.js

module.exports = {
    "files": ["*html","*js","*css"],
    "server": true,
    "port": 3000,
    "routes": {
      "/foo": "foo"
    }
};

Checkout all the options available in the docs.

For this example, I have also included an entry point at index.html, and an additional html file to try a new route:

<!-- index.html -->
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>INDEX</title>
  </head>
  <body>
    <p>
      This is the index
    </p>
  </body>
</html>

<!-- foo/index.html -->
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>FOO</title>
  </head>
  <body>
    <p>
      This is a foo
    </p>
  </body>
</html>

Now that you have a sample project outlined, start the server.

3. Start the web server

$ browser-sync start --config bs-config.js

Browser-sync will open your default web browser and load the index.html file:

Test out the router by navigating to localhost:3000/foo, and you should see the content of foo/index.html

Serving a local site is just that quick. If you don’t know where to start with your bs-config.js file, you can have browser-sync generate one for you:

$ mkdir testserver && cd testserver
$ browser-sync init
[BS] Config file created bs-config.js
[BS] To use it, in the same directory run: browser-sync start --config bs-config.js

In the generated config file, you find all the options that are used internally, but there are many more described on the website:

module.exports = {
    "ui": {
        "port": 3001,
        "weinre": {
            "port": 8080
        }
    },
    "files": false,
    "watchOptions": {},
    "server": false,
    "proxy": false,
    "port": 3000,
    "middleware": false,
    "serveStatic": [],
    "ghostMode": {
        "clicks": true,
        "scroll": true,
        "forms": {
            "submit": true,
            "inputs": true,
            "toggles": true
        }
    },
    "logLevel": "info",
    "logPrefix": "BS",
    "logConnections": false,
    "logFileChanges": true,
    "logSnippet": true,
    "rewriteRules": [],
    "open": "local",
    "browser": "default",
    "cors": false,
    "xip": false,
    "hostnameSuffix": false,
    "reloadOnRestart": false,
    "notify": true,
    "scrollProportionally": true,
    "scrollThrottle": 0,
    "scrollRestoreTechnique": "window.name",
    "scrollElements": [],
    "scrollElementMapping": [],
...

For example: to test your application over https, set "https": true right beneath your server options. When browser-sync opens the browser with your entry point, the browser should warn you that the connection isn’t safe:

Click on Proceed to localhost (unsafe) to load your content.

Easy peasy.

Service workers

Sat, 15 Oct 2016 00:00:00 GMT

A service worker is a script that sits between a web page and the network, acting as a proxy for network requests: if the network is available, the request is passed on to the target url; if the network is not available, the service worker will handle the request itself by checking cached responses or or queuing the request for synchronization once the network is available. Service workers are bridging the gap between native mobile applications and traditional web applications that require a network connection to function. Sometimes called progressive web apps, these applications enable a seamless offline experience when the user is not connected to a network.

Service workers can be used for data synchronization in the background, but data fetching and sync-ing are only one of the ways we can leverage this API. We can:

use background resources to perform client-side compilation of assets (useful for web development)
pre-fetch data that the user is likely going to need in the near future (think photo albums or playlists)
share large data sets between multiple pages
notify mobile users of updates via push notifications

For example, The Washington Post uses service workers to send push notifications of breaking news content to users of their web application on android.

How do they work?

Service workers rely on the Promise API — a Promise is a proxy for “a value that may be available now, or in the future, or never.” They also use the event emitter pattern, where listeners are registered to handle specific events, such as fetch requests.

To employ a service worker you must first register a script with the service worker API: navigator.serviceWorker.register('./sw.js'). Checkout the following example (get the gist):

<!-- index.html references the javascript application -->
<html>
  <body>
    <h1>Service worker test app</h1>
    <script src="./app.js"></script>
  </body>
</html>

// app.js
.register('sw.js') // register a worker script with the service
navigator.serviceWorker
  .then(function(reg) {
    // do the following if registration is successful
    if(reg.installing) {
      // first visit or fresh reload of the worker
      console.log('Service worker installing on registration.');
    } else if(reg.waiting) {
      // a worker may be installed but waiting for another worker to stop
      console.log('Service worker already installed on registration.');
    } else if(reg.active) {
      // pages from the same origin may share a service worker
      console.log('Service worker already active on registration.');
    }
  }).catch(function(error) {
    // if registration is unsuccessful, tell us why
    console.log('Registration failed with ' + error);
  });

// sw.js
console.log('inside registration script')

this.addEventListener('install', function(event) {
  console.log('Service worker installed.')
});

this.addEventListener('activate', function (event) {
  console.log('Service worker activated.')
})

Service workers can be registered with a specific origin and path to watch. Since they run in a background thread with a context separate from the application javascript, service workers have no DOM access. So don’t try to perform synchronous work (like accessing LocalStorage) or update the DOM from a service worker.

Running this example code sends some output to the console:

NOTE: A single service worker can control many pages (so be careful with global variables).

Developing with service workers

Service workers can only function in a secure context, which means that you can only register a service worker from a web application that is served over https. One option is to use GitHub pages to deploy your application, as these pages are generally served over https. Another option is to use a local web server (localhost is considered semi-secure).

I use Browser-sync, a super light-weight easy to start web server. In my project directory, I start the browser-sync server and set it to watch my javascript files:

$ npm install -g browser-sync
$ browser-sync start --server --files *.js
[BS] Access URLs:
 ----------------------------------
       Local: http://localhost:3000
    External: http://10.0.0.8:3000
 ----------------------------------
          UI: http://localhost:3001
 UI External: http://10.0.0.8:3001
 ----------------------------------
[BS] Serving files from: ./
[BS] Watching files...

Browser-sync is so helpful, it will even open a browser window for you. :awthanks:

Don’t forget to checkout the Chrome developer tools, which can be super helpful in creating different states for your service worker

MOAR Resources

For more about the service worker API, checkout these resources:

the W3C spec
the MDN docs
and the example https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API/Using_Service_Workers

The EventEmitter Pattern and the Event Loop -- a timeline

Fri, 14 Oct 2016 00:00:00 GMT

The EventEmitter pattern is a derivative of the Observer pattern: an object can notify a set of observers when a change in its state occurs. Consider the following example using the EventEmitter pattern:

const EventEmitter = require('events').EventEmitter
const fs = require('fs')

const findPattern = (files, regex) => {
  const emitter = new EventEmitter()

  files.forEach(file => {
    fs.readFile(file, 'utf8', (err, content) => {
      if (err) return emitter.emit('error', err)

      emitter.emit('fileread', file)

      let match
      if (match = content.match(regex)) {
        match.forEach(elem => {
          emitter.emit('found', file, elem)
        })
      }
    })
  })
  return emitter
}

// fileA.txt contains the words "hello blah blah blah"
// fileB.json is empty
// fileC.md does not exist
const files = ['fileA.txt', 'fileB.json', 'fileC.md']

findPattern(files, /hello \w+/g)
  .on('fileread', file => {console.log(`${file} was read`)})
  .on('found', (file, match) => {console.log(`Matched ${match} in file ${file}`)})
  .on('error', err => {console.log(`Error emitted: ${err.message}`)})

Some of the script is executed synchronously, but the real work is done asynchronously. The EventEmitter pattern takes advantage of the event loop to work efficiently.

Keep the following schematic in mind as you parse through the execution of this script:

image credit: Mario Casciaro and Luciano Mammino from Node.js Design Patterns. Colour notations are mine.

The first set of operations happen synchronously:

call findPattern with the file list and regex pattern
create a new event emitter
submit readFile I/O request for fileA.txt to the event demultiplexer
submit readFile I/O request for fileB.json to the event demultiplexer
submit readFile I/O request for fileC.md to the event demultiplexer
return the event emitter
register fileread listener on the event emitter
register found listener on the event emitter
register error listener on the event emitter

Once the I/O operations are submitted to the event demultiplexer, they will return in the order in which the requests are fulfilled. In this example, fileC.md does not exist, so the event demultiplexer submits the result along with the handler to the Event loop.

The event loop runs the result through the callback for fileC.md, which emits an error event (line 9)! The EventEmitter adds the error handler to the event loop.

In the meantime, the event demultiplexer has fulfilled the I/O request for fileB.json, and added the result and the callback to the event loop.

The event loop runs the result through the callback for fileB, which emits a fileread event (line 11). The EventEmitter adds the fileread handler to the event loop.

The event demultiplexer also fulfills the I/O request for fileA.txt and adds the result and the callback to the event loop.

The event loop runs the result through the callback for fileA, which emits a fileread event (line 11) AND a found event (line 16). The EventEmitter adds the fileread and found handlers to the event loop.

The event loop is busily processing all of these events in the order of arrival:

run the listener callback for the error event (fileC)
run the listener callback for the fileread event (fileB)
run the listener callback for the fileread event (fileA)
FINALLY — run the listener callback for the found event (fileA)

The result of running this script on the command line:

$ node --use_strict events.js
Error emitted: ENOENT: no such file or directory, open 'fileC.md'
fileB.json was read
fileA.txt was read
Matched hello blah in file fileA.txt

Side note: set the flag --use-strict to run scripts using ES6 syntax

Why did the files process in what seems to be reverse order? The event demultiplexer adds work to the event loop as it fulfills I/O requests. When readfile() is given three files — a file with content (fileA), an empty file (fileB), and a non-existent file (fileC) — it is reasonable to assume that the I/O request for a non-existent file will return first, as there is no file to read. It is also reasonable that an empty file will return next, as there is very little work to be done reading an empty file. Finally, the I/O request for the file with content returns — it requires more work to read a file with content than an empty file.

For more about the event emitter pattern in Node, check out Node.js Design Patterns by Mario Casciaro and Luciano Mammino.

callbacks and the event loop

Wed, 12 Oct 2016 00:00:00 GMT

When I started writing javascript in earnest a couple of years ago, I was stuck in a very procedural way of thinking — all of my functions executed synchronously. If you replaced function with def and removed all var keywords, you would be reading Ruby (true story: yesterday I was debugging an issue with a React module and put in a binding.pry instead of debugger :facepalm: Old habits die hard).

Last year, I joined a team that writes javascript in the Node.js style, using callbacks for flow control. While I became familiar with the pattern pretty quickly, the whys and wherefores were missing from my understanding. Today I am filling in the gaps by reading Node.js Design Patterns by Mario Casciaro and Luciano Mammino.

The Event Loop

The first step to understanding asynchronous javascript is to understand the event loop. It is a single thread that processes one message at a time, as they appear in the FIFO event queue.

image credit: Mario Casciaro and Luciano Mammino from Node.js Design Patterns. Colour notations are mine.

An I/O request is sent to the event demultiplexer.
The event demultiplexer queues the I/O instructions along with the context and a callback function that contains the instructions for what do to with the result once it is received.
When the I/O order is filled, the event demultiplexer pops an event on to the event queue, with the filled I/O request, and the callback handler.
The event loop pulls the event off of the event queue and applies the result to the callback function.
The event loop either sends the result of the callback operation to the application, or generates more I/O work for the event demultiplexer.

When I check my email in the morning, I go through a similar process:

My inbox is the event demultiplexer, where requests for resources are made.
I (as the event loop) process emails from in the queue from oldest to newest, and I only process one at at time.
For each message I read the context and perform one of two actions (callback):
- Process the message and return to the queue to pick up another message.
- Add a new action to my todo list event queue along with the a reminder to attach the result of the new action to the email when I respond (callback).

When I am finished with my email inbox I start on my todo list, performing the actions required and responding to the email. Sometimes the action will generate additional actions, which I append to the end of my todo list. While the idea of an infinite todo list is a little disheartening to a human, this is how the V8 javascript engine was designed: it performs the work with maximum efficiency, and can do so with only a single thread.

The difference between synchronous and asynchronous javascript

Synchronous javascript can be easily identified by the use of a return statement, which returns control back to the caller.

function introduction (firstName, lastName) {
  return 'Hello, my name is ' + firstName + ' ' + lastName
}

introduction('Keighty', 'Leonard')
// "hello! my name is Keighty Leonard"

Asynchronous javascript uses a ‘continuous passing pattern’, where the result of a function is passed to a handler for further processing.

function createEmailAddress (firstName, lastName, callback) {
  // create asynchronous call
  console.log('before the async')
  setTimeout(function () {
    // do some work
    var email = firstName.toLowerCase() + '.' + lastName.toLowerCase() + '@example.com'
    console.log('during the async')
    // pass the result of the work to the callback
    callback(email)
  }, 2000)
  console.log('after the async')
}

var cb = function (result) {
  console.log(result)
}

createEmailAddress('Keighty', 'Leonard', cb)

// > before the async
// > after the async
// > during the async
// > keighty.leonard@example.com

createEmailAddress is called. When execution reaches the setTimeout function, and the 2000ms have elapsed, the callback passed to setTimeout is placed in the event queue, awaiting its turn for execution. The thread is release and does not have to wait for the setTimeout period to elapse before continuing execution — it continues with the second console statement (‘after the async’). When the setTimeout callback is finally processed, the firstName and lastName variables are still accessible because they exist in the function closure.

Javascript has many features that make the continuous passing pattern easy:

Closures allow you to access the environment on which a function was created, no matter when the callback is invoked.
Functions are first class data types, meaning they can be assigned to variables, passed as parameters, and stored in data structures.

More on the event loop and event-emitter observer pattern to come!

git squash to speed up your workflow

Mon, 12 Sep 2016 00:00:00 GMT

I am a big fan of work-in-progress (WIP) commits. When I am working through a prototype or a spike, and need to do some experimentation with the code, I use WIP commits like they are save points in a game. These points act like milestones, and allow me to keep working with the confidence that I can always go back to an earlier working state. I use these aliases for the job:

# in ~/.bashrc
$ alias wip="git commit -am 'WIP'"
$ alias popwip="git reset --soft head~1; git reset head ."

When I get the work into a good state, I wip it (I may even sing 🎶 When a problem comes along… you must whip it! 🎶). If I want to grab the last WIP commit from the branch, I popwip to pop the last commit off the stack and put the last bunch of changes back into my working directory.

However, when I have been prototyping for a while, making wip commits as I go, popping a bunch of commits off the stack gets a little unwieldy. Until now I used an interactive rebase to rewrite my branch’s commit history. For example:

$ git log --pretty=format:\"%h %ad | %s%d [%an]\" --graph --date=short

For this git history, I would rebase on sha 83bfc47:

$ git rebase -i 83bfc47

pick 540c0fe WIP
pick 8af4cfe WIP
pick 50bd7db WIP
pick f90410b WIP

# Rebase 83bfc47..f90410b onto 83bfc47
#
# MOAR git-structions below...

I would proceed to reword the first and fixup the rest, writing a thoughtful and all-encompassing commit message to convey the intention behind this chunk of work. Then I discovered git merge --squash

$ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.

$ git merge --squash test_branch
Updating 83bfc47..f90410b
Fast-forward
Squash commit -- not updating HEAD
 routes/performance.js                          | 3 +++
 routes/performer.js                            | 3 +++
 routes/user.js                                 | 2 ++
 src/app.js                                     | 3 +++
 src/components/performance-frame.jsx           | 3 +++
 src/containers/filterable-performance-grid.jsx | 3 +++
 6 files changed, 17 insertions(+)

$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

	modified:   routes/performance.js
	modified:   routes/performer.js
	modified:   routes/user.js
	modified:   src/app.js
	modified:   src/components/performance-frame.jsx
	modified:   src/containers/filterable-performance-grid.jsx

All of the changes I have made on that branch since it diverged from master have been added to the index. I am free to either chunk up the changes into different commits (with commit messages that are more descriptive than WIP), or add them all to a single commit.

GOODBYE rebase -i! Hello merge --squash!

importing and exporting: modular React

Thu, 18 Aug 2016 00:00:00 GMT

While working on rewriting my side-project (VirtualPlaybill) in React, I got hung up on a very basic plumbing concept: named exports.

With ES5 you can export in three ways and import in one:

// EXPORT STYLE 1
module.exports = {
  crunchy: function () { console.log('CRUNCHY!') },
  bacon: function () { console.log('BACON!') }
};

// EXPORT STYLE 2
exports.crunchy = function() { console.log('CRUNCHY!') }
exports.bacon = function() { console.log('BACON!') }

// EXPORT STYLE 3 (and my personal favourite)
var breakfast = {
  crunchy: function () { console.log('CRUNCHY!') },
  bacon: function () { console.log('BACON!') }
}
module.exports = breakfast

An export is a meaningful bundle of code. Importing takes a single form, providing a handle to the exported bundle:

// IMPORT
var breakfast = require('breakfast.js')

breakfast.crunchy() // CRUNCHY!
breakfast.bacon() // BACON!

In ES6 there is a lot more flexibility: exports can be named and imports can take many forms.

Named exports

Importing a named export allows you to grab only the functions you need from a module, leaving the rest behind. Use curly braces to import a named export:

//-----test.js-----//
export crunchy = () => { console.log('CRUNCHY!') }
export bacon = () => { console.log('BACON!') }

// IMPORT BY NAME
//-----main.js-----//
import { crunchy } from './test'
crunchy() // CRUNCHY!
bacon() // Uncaught ReferenceError: bacon is not defined(…)

Default exports

In addition to named exports, you can specify one default export per file. Simply importing that file provides a reference to the default export:

//-----test.js-----//
export crunchy = () => { console.log('CRUNCHY!') }
default export bacon = () => { console.log('BACON!') }

// IMPORT BY DEFAULT
//-----main.js-----//
import myFunc from './test'
myFunc() // BACON!

You can also import both default and named exports at the same time:

import myFunc, { crunchy } from './test'
crunchy() // CRUNCHY!
myFunc() // BACON!

For even more import options, checkout the section on imports from the MDN docs:

import defaultMember from "module-name";
import * as name from "module-name";
import { member } from "module-name";
import { member as alias } from "module-name";
import { member1 , member2 } from "module-name";
import { member1 , member2 as alias2 , [...] } from "module-name";
import defaultMember, { member [ , [...] ] } from "module-name";
import defaultMember, * as name from "module-name";
import "module-name"

name

Name of the object that will receive the imported values.

member, memberN

Name of the exported members to be imported.

defaultMember

Name of the exported default to be imported.

alias, aliasN

Name of the object that will receive the imported property

module-name

The name of the module to import. This is a file name.

What you CAN’T do…

… is use ES5 syntax and expect the import to know what you mean:

//------my-component.js-------//
class MyComponent extends React.Component {
  render() {
    return <p>this is an awesome react component </p>
  }
}

export default MyComponent

//------main.js-------//
import MyComponent from './my-component'
console.log(MyComponent) // undefined

I was trying to write ES5 in ES6! Without an explicit export in front of the class declaration, my default had no idea what I intended to export, so it did nothing. Adding export to the class definition, or even moving the whole export default to the class declaration does the job:

//------my-component.js-------//
default export class MyComponent extends React.Component {
  render() {
    return <p>this is an awesome react component </p>
  }
}

//------main.js-------//
import MyComponent from './my-component'
console.log(MyComponent) // function MyComponent() {...}

Thanks to this stackoverflow answer for pointing me in the right direction, and exploringjs for even greater detail!

nested routes with expressjs

Tue, 12 Apr 2016 00:00:00 GMT

Whenever I need to build a quick web application I turn to ExpressJS. It is a fast, minimal, easy to configure web server that puts the E in MEAN). I wanted to build a REST API for a hobby app and found that the docs for how to nest routes are relatively few (see blog, stackoverflow answer). Combining these two resources, I learned a simple method for keeping your routes separate while creating a nested routing structure.

The Plan

I want to create routes that look like these:

/foo
/foo/bar
/foo/42
/foo/42/baz
/foo/42/baz/123

Using the express-generator and a few simple commands, you can get your server up and running fast:

$ npm install express-generator -g
$ express myapp
$ cd myapp
$ npm install
$ npm start

You will end up with a working directory that looks like this:

├── app.js
├── bin
│   └── www
├── node_modules
│   ├── ...
├── package.json
├── public
│   ├── images
│   ├── javascripts
│   └── stylesheets
│       └── style.css
├── routes
│   ├── index.js
│   └── users.js
└── views
    ├── error.jade
    ├── index.jade
    └── layout.jade

Go ahead and delete the /routes/users.js file and the following lines from app.js

...
var users = require('./routes/users');
...
app.use('/users', users);

Then create a new routes file named /routes/foo.js and and require it from your app server.

...
app.use('/foo', require('./routes/foo'));
...

How to `GET /foo`

Inside /routes/foo.js, you need to require the router module from ExpressJS, define a route on it, and export it back out.

var express = require('express')
var router = express.Router()

// GET /foo
router.get('/', function (req, res, next) {
  res.send('this is the index for foo')
})

module.exports = router

Everytime you change a route, you need to restart your server. Nevigating to localhost:3000/foo should display the message:

How to `GET /foo/bar`

Create simple nested routes by requiring the child route file from the parent route file.

var express = require('express')
var router = express.Router()
var bar = require('./bar')

// GET /foo
router.get('/', function (req, res, next) {
  res.send('this is the index for foo')
});

// GET /foo/bar
router.use('/bar', bar) // tell the router to use bar.js for child routes

module.exports = router

Create a new routes file named /routes/bar.js and define a root route the same way you did for /routes/foo.js:

var express = require('express')
var router = express.Router()

// GET /foo/bar
router.get('/', function (req, res, next) {
  res.send('this is the index for bar')
});

module.exports = router

Bounce your server again and navigate to localhost:3000/foo/bar

How to `GET /foo/42`

Expecting a parameter? No problem! access URL params directly from the request object: req.params.nameOfParam:

...
// GET /foo/42
router.get('/:number', function (req, res, next) {
  res.send('this is foo #' + req.params.number)
})

module.exports = router

Navigate to localhost:3000/foo/42 to see the result.

How to `GET /foo/42/baz`

Getting a child route from a parameterized parent is where I was getting confused. It turns out that you need to configure the router a little differently when you are passing params through.

...
var baz = require('./baz')

...
// GET /foo/42/baz
router.use('/:number/baz', baz)

module.exports = router

You need to pass an options object to express.Router to merge the params from any parent route:

var express = require('express')
var router = express.Router({mergeParams: true}) // don't forget the parent params!

// GET /foo/42/baz
router.get('/', function (req, res, next) {
  // the param name is from the parent as well
  res.send('this is the baz for foo#' + req.params.number);
})

module.exports = router

Navigate to localhost:3000/foo/42/baz to enjoy your success.

How to `GET /foo/42/baz/123`

Parameterizing a child route is the same process as for the parent route:

var express = require('express')
var router = express.Router({mergeParams: true})

// GET /foo/42/baz
router.get('/', function (req, res, next) {
  res.send('this is the baz for foo#' + req.params.number);
})

// GET /foo/42/baz/123
router.get('/:id', function (req, res, next) {
  res.send('baz #' + req.params.id + ‘ for foo #’ + req.params.number)
})

module.exports = router

Navigate to localhost:3000/foo/42/baz/123 to enjoy your success.

TADA! a model of nested routes that you can apply to any application.

preflight check with CORS

Tue, 29 Mar 2016 00:00:00 GMT

Modern web applications can draw resources from anywhere on the web. Fonts, JavaScript libraries, images, and other data can be fetched from CDNs, Amazon, IMDB, or anywhere else that provides a public API. Early browsers restricted web applications to same-origin requests, which prevented the sharing of resources between applications, but also ensured that data from one application could not be tampered with by another. The thinkers at the W3C came up with a means of communicating safe cross-origin requests that would allow even destructive remote actions to be performed, as long as the server consented to receive the request. Enter CORS, and preflighting.

What is CORS?

Cross-Origin Resource Sharing occurs when JavaScript on a web page requests data from (or sends data to) a location on a different host. Any website that embeds a video from YouTube, uses a custom font, or posts to social media on your behalf, is making a CORS request. CORS is a set of web standards developed to enable safe cross-domain communication.

What is preflighting?

CORS functions through the specification of new HTTP headers that allow servers to describe the origin and nature of the request. For each outgoing request to a different domain, the browser will look at the headers to determine if the request that looks like it will affect data on the receiving server. Any simple request that uses one of the common HTTP verbs and a basic Content-Type are allowed to pass without further comment. But if the request has different header values, the browser checks with the server first to make sure the request is expected. This is called a pre-flight check.

What does a simple CORS request look like?

For simple requests (no pre-flight check), the method must be one of: - GET - HEAD - POST

The only extra headers allowed are: - Accept - Accept-Language - Content-Language - Content-Type

For the Content-Type header, the only allowed values are: - application/x-www-form-urlencoded - multipart/form-data - text/plain

XMLHttpRequests (aka XHRs or ajax) use CORS to mitigate the risks of pulling or pushing sensitive data across domains. For example:

var body = 'very safe, ordinary text content'
var url = 'http://api.example.com'
var request = new XMLHttpRequest()
request.open('POST', url, true)
request.setRequestHeader('Content-Type', 'text/plain')
request.send()

The browser checks out the request, sees that it is unlikely to have any negative consequences, and passes it along to the requested endpoint. In the Network tab, simple CORS requests will appear by themselves:

What does a pre-flight CORS request look like?

For non-simple requests, the method could be any of the HTTP verbs along with any other combination of headers. For example:

var body = 'possibly unstable or malicious content'
var url = 'http://api.example.com'
var request = new XMLHttpRequest()
request.open('POST', url, true)
request.setRequestHeader('Content-Type', 'application/json')
request.send()

In this request I am trying to POST a JSON object to an API endpoint, which could potentially have destructive consequences for data on the other side of the endpoint. When the browser inspects the headers of this request it gets suspicious, and fires off a preliminary request to the endpoint with the meta data from the request:

The server at the endpoint responds to the pre-flight request with a list of authorized headers, including acceptable request origins, content, and actions.

Ignoring for a moment that the requests fail (because api.example.com does not exist), the non-simple CORS request produces two requests instead of one:

The result of the pre-flight OPTIONS request will determine if the second is ever sent on to the endpoint. This is how browsers keep your application data secure from malicious or unintended changes.

Why OPTIONS, and not POST?

I found this great answer on stackoverflow that explains why the creators of CORS made a new request type, OPTIONS. Before the CORS standards were introduced, neither browsers nor servers knew how to handle cross-domain requests. Browsers would throw an error, but servers would process the request without concern. Inventing a new method type that CORS-enabled browsers AND CORS-aware servers could handshake with ensures that both sides are sensible to the meaning of the pre-flight request.

RESOURCES

Learn more about the web standards at W3C
Find out more about how CORS is implemented at Mozilla
Find out what web browsers support CORS at caniuse.com

keighty's 2015 - by the numbers

Sat, 26 Dec 2015 00:00:00 GMT

2015 has been a very eventful year. Not only did I write and launch an ebook about my code school experience, but I have also built and launched a book site, competed in a national Bridge tournament in New Orleans, celebrated 10 years of wedded bliss, spent 3 weeks traveling in the Mediterranean, and joined a new team at work where I get to deep-dive into JavaScript and browsers. Checkout my infographic for more!

RESOURCES

Interested in making your own infographic? Try canva

faster asset loading with Cache-Control

Fri, 27 Nov 2015 00:00:00 GMT

My side project (Virtual Playbill) is an image-heavy application, and I used YSlow to find a few quick performance wins. The first ‘F’ I got was for No Expires Headers:

What is an Expires Header?

Loading the page requires slow and expensive network calls to download all the JavaScript, CSS, and image files. If someone visits the page more than once, I can avoid using the network by storing a local copy of the files in their browser cache — temporary local storage that is designed for quick retrieval. I can tell the browser to cache a copy of files that don’t change very often by setting Expires headers.

When a web server receives a request it adds some metadata to the response in the form of HTTP headers. Headers are key-value pairs that tell the browser information about the file it has received, such as how big it is (Content-Length), what kind it is (Content-Type), and how long the browser should cache the response (Cache-Control).

In the case of Virtual Playbill, all the images are served from AWS Simple Storage Service (S3), and there are a couple of ways to add headers.

1. Tedious Manual Addition

You can add headers manually to a single asset stored in S3 using the metadata section:

But I have more than a hundred files to change (yawn).

2. Write a Nodejs Script using `aws-sdk`

var aws = require('aws-sdk')
var s3 = new aws.S3();

var params = {
  Bucket: 'Your_bucket_name',
}

new Promise(function (resolve, reject) {
  s3.listObjects(params, function(err, data) {
    if (err) console.log('Something went wrong when retrieving the list of objects: ' + err)
    resolve(data.Contents.map(function (image) { return image.Key }))
  })
}).then(function (imageList) {
  return collectImageParams(imageList)
}).then(function (imageChangeParams) {
  imageChangeParams.forEach(function (params) {
    s3.copyObject(params, function (err) {
      if (err) console.log("Oops! Something went wrong with copy/replace for " + params)
    })
  })
})

// Duplicating a file in S3 will change the metadata.
// You can make sure the new file has the same
// access control by passing through the right permissions
// and ContentType along with the new CacheControl header
function collectImageParams(imageList) {
  return imageList.map(function (imageName) {
    var imageParams = {
      'Bucket': params.Bucket,
      'ACL': 'public-read',
      'MetadataDirective': 'REPLACE',
      'CacheControl': 'max-age=2592000',
      'ContentType': 'image/jpeg'
    }
    imageParams.Key = imageName
    imageParams.CopySource = '/' + imageParams.Bucket +'/' + imageParams.Key
    return imageParams
  })
}

Still.. I don’t want to have to run this script every time I upload a new file (yawn)…

3. Add Headers Every Time You Upload

S3Upload.prototype.uploadToS3 = function(file, url, public_url) {
  var this_s3upload, xhr;
  this_s3upload = this;
  xhr = this.createCORSRequest('PUT', url);
  ...
  xhr.setRequestHeader('Content-Type', file.type);
  xhr.setRequestHeader('x-amz-acl', 'public-read');

  // Set the Cache-Control header for every file upload
  xhr.setRequestHeader('Cache-Control', 'max-age=2592000');
  return xhr.send(file);
};

Great Results

No matter how you add the Cache-Control header, the result is amazing:

Resources

For more information about HTTP headers and the browser cache, check out these useful resources:

javascript object property meta properties

Sat, 07 Nov 2015 00:00:00 GMT

JavaScript Objects are complex types, and even defined properties have a control panel of their own. You can view the content of the control panel using Object.getOwnPropertyDescriptor(), and flip their switches using Object.defineProperty().

Note: These property descriptors are for own properties — that is, properties that are directly defined on an object, not those that come along for the ride on the object’s prototype chain.

The property descriptor holds meta data

> var me = {
  name: "Katie"
}

> Object.getOwnPropertyDescriptor( me, "name" )
{
  value: "Katie", // the value associated with the property
  writable: true,   // true if the value of the property can be changed
  enumerable: true, // true if the property shows up during enumeration of the properties
  configurable: true // true if any of these meta properties can be changed
}

Change the property value

Object.defineProperty( me, "name", {
  value: "keighty"
})

me.name // "keighty"

Make the property read only

Object.defineProperty(me, "name", {
  writable: false
})

me.name = "k80"
me.name  //"keighty" -- actual value of me.name did not change

Make the property hidden from for-loops (or any enumeration)

me.hobbies = ["bridge", "curling", "tap dancing"]
me // Object {hobbies: Array[3], name: "keighty"}

for(fact in me) {
 console.log(fact)
}
// All the properties defined on me are written to the console:
// name
// hobbies

Object.defineProperty(me, "name", {
 enumerable: false
})

for(fact in me) {
 console.log(fact)
}
// Actually.. only enumerable properties are written to the console:
// hobbies

Freeze the property

Object.defineProperty(me, "name", {
 configurable: false
})

Object.getOwnPropertyDescriptor(me, "name")
// { value: "keighty",
//   writable: false,
//   enumerable: false,
//   configurable: false }

Object.defineProperty(me, "name", {
 writable: true
})

Uncaught TypeError: Cannot redefine property: name(…)

From the docs: “The Property Descriptor type is used to explain the manipulation and reification of named property attributes. Use Object.defineProperty for fine-grained control over the visibility and mutability of your object’s properties.

estimate != commitment != target

Sat, 31 Oct 2015 00:00:00 GMT

Software estimation is hard, but I didn’t really understand why until I started reading Software Estimation: Demystifying the Black Art by Steve McConnell. It was published in 2006, and is very much still relevant — there are more ways to do estimation wrong than to do it right.

The biggest “aha” moment for me was learning the difference between estimates, targets, and commitments. I always thought that estimates were commitments and treated them like targets. If a task took longer than my estimate, I would worry that I had blown the project plan. The source of my stress was really semantics, and a deep misunderstanding of project plans in general.

1. An estimate is a range where there is a 90% chance of finding the completion date

An estimate is not a single point in time — that is a target. There is uncertainty in software development, and uncertainty about the uncertainty.

There is a limit to how well a project can go but no limit to how many problems can occur.

Estimates are probability distrubutions of possible outcomes:

Most of my experience in estimating projects has been to break a feature down into small tasks and estimate those. Tightly defined requirements for a task make it easier to estimate, but there are always complications that can throw things off track. Thinking about an estimate as a range of possibilities between the best and worst case scenarios reduces stress about being right the first time.

2. A commitment is a promise to deliver defined functionality at a certain level of quality at a certain date.

You can use estimates to plan commitments, but not the other way around. If ten features take a year to develop, but your customer needs to have something in six months, you can commit to delivering four or five features by that target, but there is no way you can deliver ten.

3. A target is a single point in time, and is usually determined by business needs.

A target has more in common with a commitment than an estimate — it can have no bearing on the time estimated to complete a project, but will often define the envelope of time. Like in the previous example, the target for delivery to the customer was six months, but the commitment could be four or five features, depending on the estimates for each feature.

Estimation must come before planning, and before you really understand all of the requirements for the project. More often than not, events that happen during the project will have a big impact on the assumptions made to estimate the project at the beginning. Estimates are hard, but McConnell has some awesome advice on how to get it right.

The Clean Coder

Sat, 26 Sep 2015 00:00:00 GMT

In The Clean Coder: A Code of Conduct for Professional Programmers, Bob Martin (Uncle Bob) tells you what it means to be a professional programmer. Much of what distinguishes a professional from a non-professional can be reduced to responsibility: a professional is responsible for their project, their team, and their own career.

Protect your projects

Your first responsibility, according to Uncle Bob, is to your employer, which means doing the job that you have been hired to do.

Make reasonable estimates. Your project does not exist in isolation from other parts of the company. An estimate is a probability distribution of likely outcomes, not a single point in time. Many stakeholders rely on your estimates for coordinating marketing, sales, and other downstream tasks.
Meet your commitments. Estimates are different from commitments. If you have committed to delivering a feature or a product by a target date, meet that date.

If it can’t be done, you can still commit to actions that will bring you closer to the target. Finding out if it can be done can be one of the actions to commit to!

Protect your time. An easy way to ensure the success of your project is to devote the time and energy you have already promised in your estimates. Meetings can be disruptive to your workflow, and you should decline any meeting that does not move your project forward. If you can’t decline, ask for an agenda and clear goals before you accept.

Sometimes the meeting will be about something that interests you, but is not immediately necessary. You will have to choose whether you can afford the time. Be careful — there may be more than enough of these meetings to consume your days.

Protect your team

A team-player communicates frequently, keeps an eye out for his or her teammates, and executes his or her own responsibilities as well as possible.

Stand by your original estimates, and don’t confuse them with commitments. Your estimates should protect your team from long hours, undue management pressure, and unpleasant surprises. Don’t promise what your team can’t deliver.

Business likes to view estimates as commitments. Developers like to view estimates as guesses. The difference is profound.

Mentor less experienced engineers. Contributing to the growth of everyone will make your team stronger. Don’t wear your headphones to cut yourself off from interruptions. Making yourself available as a resource will get someone else on the team unstuck, which moves your project forward (Uncle Bob recommends using TDD instead of the Zone to pick up the stack if you have to drop it). “It is unprofessional to be a loner or a recluse on a team.”

Protect your career

Your employer is not responsible for keeping your skills up to date. They are not required to ensure that you are aware of cutting edge technologies and languages.

Your career is your responsibility. It is not your employer’s responsibility to make sure you are marketable. It is not your employer’s responsibility to train you, or to send you to conferences, or to buy you books. These things are your responsibility. Woe to the software developer who entrusts his career to his employer.

Practice coding. Uncle Bob is a big proponent of practicing code. Performing code kata before starting work will put you in the right frame of mind.

Professionals practice. True professionals work hard to keep their skills sharp and ready. It is not enough to simply do your daily job and call that practice. Doing your job is daily performance, not practice. Practice is when you specifically exercise your skills outside of the performance of your job for the sole purpose of refining and enhancing those skills.

Learn about the industry. Falling behind in new technologies will make you obsolete. You will not stay marketable to new employers (or even your current employer), if you do not stay current.

Reading The Clean Coder was like sitting down with a mentor and asking how I can grow my career. I would recommend it for anyone entering the profession today!

keep your code close and your notes closer with a global gitignore

Fri, 18 Sep 2015 00:00:00 GMT

I like to keep my notes as close to the code as possible. When I start on a new code base, I create a folder at the top called aa_notes. Super obvious, and it makes an awesome scratch pad for things I don’t want to lose. So, how can you do this without checking your folder in to source control? Adding it to a local .gitignore is one strategy, but that change will also need to be checked in to git.

The answer is to use a global gitgnore.

You can configure a global gitignore file in your home directory that will ignore items in every repo on your machine.

1. Add a folder to your source control

2. Create a file and git will suggest to track it

➜  virtualplaybill2 git:(master) touch aa_notes/this_is_a_note_file.md
➜  virtualplaybill2 git:(master) ✗ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Untracked files:
  (use "git add file..." to include in what will be committed)

        aa_notes/

nothing added to commit but untracked files present (use "git add" to track)

3. Set up your global gitignore

➜  virtualplaybill2 git:(master) ✗ git config --global core.excludesfile ~/.gitignore_global
➜  virtualplaybill2 git:(master) ✗ echo "aa_notes" > ~/.gitignore_global

4. Git will ignore all folders on your machine named aa_notes

➜  virtualplaybill2 git:(master) ✗ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean

This comes in super handy when you want to make clarifying notes to yourself without sharing them with all of your collaborators. Of course, if they are good things to share, you should probably add them to the project wiki.

I keep things like daily infobits, backdoor patches, bits of scripts, and other things. Sometimes writing a note about something helps me remember it a little better, so I still keep my own notes, even if the info is already on the wiki.

Checkout this post on GitHub for this and more ideas for fine tuning what git ignores.

hunting monkey patches

Sat, 08 Aug 2015 00:00:00 GMT

Ruby monkeypatching is a dangerous, but necessary tool. It is convenient to add new behaviours to existing classes, or to replace existing methods with more customized or secure code. Once the patch is in place, however, it is easy to forget that it is there.

Recently, I was trying to discern the origin of such a patch, and discovered a new (to me) function in pry, my favourite Ruby debugger: show-source.

Put a binding pry below the monkeypatch so that it can be caught at boot time
Use cd to change directories into the reopened class
Use show-source to find the exact location where the method was originally defined
Use show-source -a to show the source and location of EVERY MONKEYPATCH

In action:

1. Debug your code at boot time

2. Reopen the class

3. Use ‘show-source’

4. Use ‘show-source -a’

Don't count your chickens if they are ActiveRecord objects

Thu, 07 May 2015 00:00:00 GMT

You may believe that #length, #size, and #count are fairly equivalent:

irb(main):002:0> yourRubyChickens =  %w{ Chantecler RedShaver RhodeIslandRed }
=> ["Chantecler", "RedShaver", "RhodeIslandRed"]
irb(main):003:0> yourRubyChickens.length
=> 3
irb(main):004:0> yourRubyChickens.size
=> 3
irb(main):005:0> yourRubyChickens.count
=> 3

But, beware of #count in Rails!

[1] pry(main)> yourRailsChickens = Chicken.all
  Chicken Load (2.1ms)  SELECT "chickens".* FROM "chickens"
=> [#<ChanteclerChicken:0x007f98eafcd6d8
  id: 1,
  type: "ChanteclerChicken",
  color: "white",
  created_at: Thu, 07 May 2015 14:36:36 UTC +00:00,
  updated_at: Thu, 07 May 2015 14:36:36 UTC +00:00>,
 #<RedShaverChicken:0x007f98e63a9428
  id: 2,
  type: "RedShaverChicken",
  color: "white",
  created_at: Thu, 07 May 2015 14:37:01 UTC +00:00,
  updated_at: Thu, 07 May 2015 14:37:01 UTC +00:00>,
 #<RedShaverChicken:0x007f98e63a9248
  id: 3,
  type: "RedShaverChicken",
  color: "black",
  created_at: Thu, 07 May 2015 14:37:25 UTC +00:00,
  updated_at: Thu, 07 May 2015 14:37:25 UTC +00:00>,
 #<RedShaverChicken:0x007f98e63a9068
  id: 4,
  type: "RedShaverChicken",
  color: "red",
  created_at: Thu, 07 May 2015 14:37:35 UTC +00:00,
  updated_at: Thu, 07 May 2015 14:37:35 UTC +00:00>,
 #<RedShaverChicken:0x007f98e63a8e88
  id: 5,
  type: "RedShaverChicken",
  color: "white",
  created_at: Thu, 07 May 2015 14:37:53 UTC +00:00,
  updated_at: Thu, 07 May 2015 14:37:53 UTC +00:00>]

[2] pry(main)> yourRailsChickens.length
=> 5
[3] pry(main)> yourRailsChickens.size
=> 5
[4] pry(main)> yourRailsChickens.count
   (0.2ms)  SELECT COUNT(*) FROM "chickens"
=> 5

#size and #length are largely equivalent, but #count issues an additional query to the database that can seriously damage performance for the unsuspecting developer.

For more info, checkout the docs for count and length.

local load balancing

Wed, 22 Apr 2015 00:00:00 GMT

You can mimic the end-user’s UI experience accurately enough by running application code locally, but what about the backend? Once your code is deployed to production, requests will be divided between dozens of servers. If there is variation in the code paths running on each server (like during a feature rollout, for example), it is useful to determine beforehand if there are any dangerous conflicts. Enter local load-balancing.

NGiNX is load balancing software that distributes requests across multiple servers, and it’s easy to get running quickly.

Installation

You can download and compile from source at NGiNX.org, or use brew install nginx on MacOS-X.

➜  api_playground  brew install nginx
...

The default port has been set in /usr/local/etc/nginx/nginx.conf to 8080 so that nginx can run without sudo.

To have launchd start nginx at login:
    ln -sfv /usr/local/opt/nginx/*.plist ~/Library/LaunchAgents
Then to load nginx now:
    launchctl load ~/Library/LaunchAgents/homebrew.mxcl.nginx.plist
Or, if you don't want/need launchctl, you can just run:
    nginx

Start the server with the command nginx.

Configuration

Your NGiNX installation added a config file: /usr/local/etc/nginx/nginx.conf. It is mostly commented configuration examples, so you can simply replace the current server configuration block with this:

/usr/local/etc/nginx/nginx.conf

upstream myapp1 {
  server localhost:3000;
  server localhost:3030; #add as many servers as you want here
}

server {
  listen 8080;
  server_name localhost;
  location / {
    proxy_pass http://myapp1;
  }
}

Reload your NGiNX server with nginx -s reload, and hit localhost:8080 to ensure you have configured it correctly.

Start your servers

NGiNX is configured and listening on ports 3000 and 3030. Now we just have to connect some servers!

Server 1: bundle exec rails server -p 3030

Server 2: bundle exec rails server -p 3000 --pid tmp/pids/server2.pid

Pro-tip: The Rails server stores its process id in a temporary file (/tmp/pids/server.pid), and will complain if you start another server unless you specify a new temporary pid file.

Check your progress by hitting localhost:8080:

Checkout the docs for more examples of nginx configuration.

what else you can do after code school

Thu, 16 Apr 2015 00:00:00 GMT

Learning to program at a code school is a great opportunity to acquire new skills, and to make your current skills relevant in tech. Becoming a software developer is the most common goal, but there is a wide variety of roles that technical people can play in software that aren’t about writing code. Whether you have left a job in customer service, teaching, or research, the skills you gained in your last career will help you fill a niche in your new one. Here are a few paths in tech that don’t involve writing software, but do require some experience with coding, frameworks, and methodologies.

Product Manager

Code School + Business Management

As a product manager (aka product owner), you decide what features to build by studying the market and learning your customers’ needs. The result of your research is a product road map and business strategy for the next year and beyond. Field experience is essential to making good business decisions.

Project Manager

Code School + Project Management

A project manager follows the product road map feature by feature. In this role you will be responsible for marshalling resources and delivering product. Excellent communication skills are a must, as is a technical familiarity with the system under construction. You need to ask the right questions, identify risks, validate estimates, and eliminate blockers.

Engineering Manager

Code School + Project Management + People Management

High-functioning teams do not manifest by chance — they are the result of deliberate choices made at the intersection of project needs, culture fit, and career growth. As an engineering manager, you don’t just assemble the team. You also schedule projects, implement (or remove) processes, grow careers, and communicate up and down the management chain. In addition to all of these tasks, you must be familiar with the job the team is performing.

Technical Writer

Code School + Written Communication Skills

Writing technical specs and documentation will require all of your technical knowledge as well as language, teaching, and research skills. The technical writing field is expected to grow enormously in the next 10 years¹, and delivering web-based product support will be par for the course. Use your new coding skills to build documentation websites and forums, as well as generate useful content.

Technical Support

Code School + Customer Service

Technical support is growing at a similar pace to technical writing. Helping users interact successfully with a product is essential to their happiness, and companies have a vested interest in troubleshooting where these interactions go wrong. You act as liaison between the end user and the development and management teams by reporting and triaging bugs, working with developers to resolve issues, and communicating with internal customers as well as external ones.

Quality Assurance Engineer

Code School + Product Knowledge

As a QA engineer, you are a product expert, not a tester. You understand the way users interact with the product and how the features interact with each other. You can reveal blind spots in design and edge cases of user experience that will catch a feature engineer unawares. Combining attention to detail with a holistic view of the system, QA engineers ensure that users continue to trust the product you are delivering.

Not everyone leaves code school with a passion for building software, but everyone does leave with the skills they need to have an awesome career in tech. No education or experience is wasted, so don’t forget to leverage your current skill set as you are learning a new one.

Bureau of Labour Statistics: Technical Writers
↩

getting your foot in the door

Sun, 12 Apr 2015 00:00:00 GMT

Many people have asked me what it is like on the other side of code school, and if there is any advice I would give to someone before they get their first programming job. Technical ability, problem solving, and communication skills are all essential to success as a developer, and your online presence is a way to demonstrate all three. Getting your foot in the door requires having something to show — keeping your foot in the door requires a little bit more.

Have something to show

Not every developer spends every waking moment working on open source projects or inventing the next big framework. We all have lives, families, and other pursuits that take priority; however, experienced programmers likely have solid work-related contributions they can point to during an interview. First-timers need some proof that they can apply what they learn. Side projects show potential employers what you are learning, and that you are interested in programming, not just the salary it comes along with.

Create a GitHub account. If you are in code school, you are likely already working towards a portfolio of apps and widgets to show off during an interview. If you are considering code school, this is something you should not wait to do. When you work through a tutorial, commit it to GitHub, give it a README, and include what you are working through and why.

Start filling your toolbox

You can usually assemble a piece of Ikea furniture with only an allen key, but what about that desk drawer that suddenly calls for a screwdriver? Completing the task requires the right tools for the job, and your ability to solve a programming problem is no different. Git, Chrome Developer Tools, and Pry are part of the the Swiss Army knife for any web developer.

Git is like “track changes” for source code. The docs describe it as a series of snapshots of your code base. “Every time you commit, or save the state of your project in Git, it basically takes a picture of what all your files look like at that moment and stores a reference to [it][^1].” Source control allows you to work in the same code base with other developers and not step on one another’s toes. Start with Git Immersion for some practical experience.
Chrome Developer Tools give you access to everything that happens to your application when it is sent to the browser. Troubleshoot Javascript, CSS, and layout issues without having to refresh the page. Check out the tutorials at discover-devtools by CodeSchool.
Pry gives you an interactive debugging session wherever you put a breakpoint in a Rails application. binding.pry throws you down the rabbit hole into the inner workings of your program’s psyche. Check out exploring the stack for more details about this powerful tool.
Use the docs. Knowing how to find and navigate documentation is more important than remembering how to do File I/O. If you find yourself in a technical interview where they don’t let you search the web for help, share your thought process, imagine the solution you might have found had you performed the search, and work your way through the problem with pseudocode. Experienced developers use Google search and StackOverflow constantly, so pop that error message into the search bar to see what turns up. A word to the wise: if an answer was posted more than two years ago, it may no longer be relevant.

Get to know your community

Your next job in programming is more likely to come from someone in your network than from hitting the street with your resume. If you can claim an acquaintance with someone from a company you want to work for, that is one more point in your favour.

Go to meetups. If you are shy about meeting lots of new people at once, try introducing yourself to the invited speakers at the end — they are usually well connected and respected in the community. If you approach them with your name and a question or comment about their talk, they are likely to remember you.

Do scary things

Not like skydiving, or alligator wrestling. Write blogs, contribute to open source projects, start your own programming language, or become visible on social media. You don’t need to be an expert to have an opinion. Don’t stop taking risks after you land your first job, either.

Find an area of your work that people are avoiding and become an expert. You will be known as the person who can do that thing we all hate, and your colleagues will value your expertise.
Take the hardest ticket from the top of the stack. Don’t sift through them until you find something you think you know how to do. Your best opportunity to stretch your skills is to do something scary and uncomfortable.

I work with a lot of people who don’t have formal computer science degrees, and while their backgrounds vary, they all share a love of problem solving and a willingness to work hard. Take responsibility for your own learning — code school will only bring you so far, and the rest is up to you.

Getting started with git
↩

What's it like to be a developer?

Sat, 11 Apr 2015 00:00:00 GMT

My manager has invited me to help her give a talk to students in a Web Development program. I have been working in the field for about 18 months, and a lot of the uncertainty is still fresh in my mind: what is it like to be a programmer? How can I get a job as a junior? What can I do now that will help me in my first job? I didn’t know many people in the industry, and was unaware of the vast meetup community¹. Answers to these questions were difficult to find online, so over the next few posts I will make an attempt.

Meetings: Important communication tools.

Programming is mostly about decision making, and not all decisions can be made alone. Questions like, what technologies to use, how you should spend your time, and how to scale, require meetings with managers and other teams.

Sprint planning is a weekly meeting where the team gets together with managers to prioritize tasks for the week. You look back on last week, and either carry over tasks you didn’t accomplish, or re-prioritize based on what has arisen in the meantime.
Standups happen daily, and should only last around 10 minutes. You each take a turn sharing what you worked on yesterday, what you are working on today, and whether you are blocked by anything. It is important to speak up when you are stuck! Sharing your blockers will usually result in help; someone may have encountered this problem before, have a new approach to suggest, or have insider information that can get you to a solution.
Demos will happen on a weekly or bi-weekly basis, and are an opportunity to share your accomplishments. While many stakeholders are involved in the project and sprint planning, they are often disconnected from the work as it is being performed. Regular demos help to keep them in the loop, and are an opportunity to explain both progress and unexpected delays.
Code reviews can be formal or informal, and allow your team to gain consensus on style, design, performance, and testing. Your code design decisions will have an impact on the project as a whole, so it never hurts to get feedback from a teammate before you ship it. Readability is more important than cleverness, and is a kindness to future maintainers.

A portion of your week will always be spent in meetings. Try to identify when your most productive hours are in the day, and keep that time free. For example, if you lump meetings around lunch (none before 11am or after 2pm) you have a solid chunk of time in the morning and the afternoon to focus effectively.

Stories: How big ideas are turned into actions.

An idea like “improve the user’s experience of widget B” is a goal, not a task. It can be broken down into stories such as: identify the problem, brainstorm solutions, design the new widget, and implement the new widget. Each of these stories are actionable: there is a clear path from start to finish. Use the single-responsibility principle in your stories as well as in your code: if you use the word AND to describe the task, that is a hint that you should have two stories instead of one. Most companies will use a task management, or ticket system such as Pivotal Tracker or Jira, where you can see what work is assigned to you, and where managers can check in on your progress.

Feature Work: What should be built?

Product managers have their eye on the market. They know what features our customers want, and they know what our competitors offer. A product manager will build a product road-map with a long schedule (think months and years, not weeks). The team will sit down with the feature requirements and a first-draft design, break it down into stories, and estimate the time/complexity of each story. Managers then apportion those estimates into chunks of work — in Agile, these chunks are laid out in sprints².

Defining work in this way lets you identify what can be done in parallel and what needs to be in place before the next task can be accomplished. It also gives project managers the information they need to make decisions about what to sacrifice in order to meet a deadline. If the project is over the time budget, and gold-plated widget A needs features 1, 2, and 3, then maybe you can ship the widget with only 1 and 2, but save 3 for later. Sometimes you have to compromise on feature completeness just to get it out the door.

Troubleshooting and Maintenance: Detective work.

Finding a bug is like being a detective: you gather clues, follow leads, and occasionally end up lost in a blind alley. A single line of code can hold multiple bugs (Pro tip: watch out for Regexes³ — they are a great place for bugs to hide). In a code base with hundreds of thousands of lines, you will inevitably spend more time hunting and fixing bugs than writing new code.

Deploying code: Sending it into the wild.

Running an application in production is not the same as running it on your computer. Your application may need to serve thousands of requests per second, and a single instance running on a single server won’t cut it. There is a whole stack of management software that you would do well to be familiar with, and having at least a passing familiarity with Jenkins, Capistrano, and bash scripting will help you when it is your turn to carry the pager.

Pager duty: Evil alarm clock.

When you write software that your customers use around the clock, you have to be there when things break, no matter what time. Most companies will institute a rotation to share the burden of vigilence around the team. Having great documentation about what to do in an emergency is essential to finding a clear path of action at 3am. As with anything, the more you prepare, the higher your confidence level will be.

The job is far from routine because of the diverse set of problems to solve. I have become comfortable grabbing a story without already knowing the answer, partly from knowing that if I get stuck I am no more than a “Hey Bryan,” or “Hey KWu,” away from someone who wants to help, even if it is at 3am (thanks again, Vince!).

If you are in Portland, Oregon, checkout Calagator. Otherwise, try Meetup.com
↩
Read more about Agile methodology
↩
Regex (short for regular expression) is a search pattern based on a sequence of characters.
↩

exploring the stack

Wed, 25 Mar 2015 00:00:00 GMT

Another brilliant Rails troubleshooting technique I have recently added to my toolbox is pry-stack_explorer as an add-on to pry.

1. Add `pry` and `pry-stack_explorer` to the development/test group of your gemfile:

group :test, :development do
  gem 'pry'
  gem 'pry-stack_explorer'
end

2. Add a binding pry anywhere in your code

3. Enter `show-stack` at the pry prompt

4. Enter `up` or `down` to explore the state of the stack

Other goodies

Pass a string to up or down and pry-stack_explorer will jump to the first frame that matches:

Frame number: 0/108

From: /Users/keighty/Documents/projects/api_playground/app/models/user.rb @ line 4 User#full_name:

    3: def full_name
 => 4:   binding.pry
    5:   "#{first_name} #{last_name}"
    6: end

[6] pry(<User>)> up instrument

Frame number: 3/108
Frame type: method

From: /usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/activesupport-4.2.0/lib/active_support/notifications.rb @ line 166 ActiveSupport::Notifications.instrument:

    162: def instrument(name, payload = {})
    163:   if notifier.listening?(name)
    164:     instrumenter.instrument(name, payload) { yield payload if block_given? }
    165:   else
 => 166:     yield payload if block_given?
    167:   end
    168: end

[7] pry(ActiveSupport::Notifications)> down view

Frame number: 1/108
Frame type: method

From: /Users/keighty/Documents/projects/api_playground/app/views/greetings/hello.html.erb @ line 1 ActionView::CompiledTemplates#_app_views_greetings_hello_html_erb___3157151947187763552_70316982999600:

 => 1: <h1>Greetings, <%= @person.full_name %></h1>
    2: <p>This is a greetings view!</p>

Pass an integer to up or down to leap over frames

[8] pry(<Class:0x007fe7eb821330>>)> up 5

Frame number: 6/108
Frame type: block

From: /usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/actionview-4.2.0/lib/action_view/renderer/template_renderer.rb @ line 54 ActionView::TemplateRenderer#render_template:

    49: def render_template(template, layout_name = nil, locals = nil) #:nodoc:
    50:   view, locals = @view, locals || {}
    51:
    52:   render_with_layout(layout_name, locals) do |layout|
    53:     instrument(:template, :identifier => template.identifier, :layout => layout.try(:virtual_path)) do
 => 54:       template.render(view, locals) { |*name| view._layout_for(*name) }
    55:     end
    56:   end
    57: end

[9] pry(<ActionView::TemplateRenderer>)> down 2

Frame number: 4/108
Frame type: method

From: /usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/actionview-4.2.0/lib/action_view/template.rb @ line 333 ActionView::Template#instrument:

    331: def instrument(action, &block)
    332:   payload = { virtual_path: @virtual_path, identifier: @identifier }
 => 333:   ActiveSupport::Notifications.instrument("#{action}.action_view", payload, &block)
    334: end

To learn even more awesome ways to use pry and pry-stack_explorer, check out the docs:

troubleshooting nested scopes in angular

Wed, 11 Mar 2015 00:00:00 GMT

Many beginner Angular tutorials start with a set of canned data, an input field, and a search filter, so it was a natural place to start on my side project, Virtual Playbill. With even a small data set, watching the DOM change based on search criteria was a thrilling achievement. In the intervening months, as features were added, rebuilt, or taken away, I had inadvertently broken the search box. The tutorials were still available, and still fairly basic, but I couldn’t get it to work. Not until I found a couple of invaluable tools for troubleshooting overlapping scopes:

1. Use the developer tools console

This handy trick will give you a console object corresponding to that element’s scope:

select a DOM element in using the chrome developer tools
enter angular.element($0).scope() into the console pane ($0 is a global variable holding a reference to the selected DOM element)

// with the search box selected
> angular.element($0).scope()

$get.h.$new.a.$$ChildScope.$$ChildScope {$$childTail: null, $$childHead: null, $$nextSibling: $get.h.$new.a.$$ChildScope.$$ChildScope,
  ...
  $parent: h
  playbillRows: Array[23]
  playbills: Array[68]
  search: Object
    title: "this is search text"

// with the playbill area selected
> angular.element($0).scope()

$get.h.$new.a.$$ChildScope.$$ChildScope { $$childTail: $get.h.$new.a.$$ChildScope.$$ChildScope, $$childHead: $get.h.$new.a.$$ChildScope.$$ChildScope, $$nextSibling: null,
  ...
  $parent: h
  playbillRows: Array[23]
  playbills: Array[68]

Things are definitely looking different between the scope containing my search box and the scope where I am laying out the view: the second scope appears to be a sibling of the first. I needed something a little more visual before it completely sunk in, though.

2. Use ng-inspector

ng-inpector is a browser extension that allows you to navigate your angular app as if it were a file directory:

I can see at a glance that the scope containing the search box (#2), is separate from the scope where I am laying out my view (the unnamed scope #5).

Because of these tools, I have learned that multiple instances of the same controller do not share scope. Using ng-router, I declared the controller for the view:

  ...
  when('/', {
    templateUrl: 'views/index.html',
    controller: 'PlaybillController'
  });
  ...

In my layout, I had declared the controller once more:

  ...
  <div ng-controller="PlaybillController">
    <form class="navbar-form navbar-left" role="search">
      <div class="form-group">
        <input type="text" class="form-control" placeholder="Search shows" ng-model="search.title">
      </div>
    </form>
  </div>
  ...

The search box was already in the context of a Playbill Controller, and I had inadvertently put in another. The solution was to simply remove the second ng-controller:

  ...
  <div>
    <form class="navbar-form navbar-left" role="search">
      <div class="form-group">
        <input type="text" class="form-control" placeholder="Search shows" ng-model="search.title">
      </div>
    </form>
  </div>
  ...

I have been growing my Ruby/Rails troubleshooting toolbox for a while now, and it is good to get one started for angular as well.

more git goodies

Sun, 08 Mar 2015 00:00:00 GMT

Becoming a git expert is essential to efficiency and confidence as a software developer. I have evolved from someone who would re-clone the repo in case of a botched merge to a savvy git handler, through:

Moving files between working directory, index, stash, and commit history
Working with multiple remotes with many contributors
Handling changes to forks, branches, and pull requests
Accidental merges from pulling upstream, and checking out pull requests or remote branches

git rebase -i <sha> # pick a sha from your git history and start squashing
git clean -f # removes untracked files from your working directory
git stash # save your idea for later
git stash pop # revive your last idea
git reflog # oh my goodness, what did I just do?

# Add this to your .bashrc (or .zshrc)
alias wip="git add .; git commit -m 'WIP'"  # saves and commits the current state
alias popwip="git reset --soft head~1; git reset head ."  # pops the top commit off the stack

# Add this to your git config
[alias]
  hist = log --pretty=format:\"%h %ad | %s%d [%an]\" --graph --date=short

$ git hist # opens a human readable history -- essential for verifying where you are:
* 2ae2be1 2015-03-08 | WIP (HEAD, origin/source, source) [keighty]
* 247f86f 2015-02-16 | Change portfolio subtitle. [keighty]
* e2a4fd0 2015-02-16 | Add facebook and google+ sharing options. [keighty]
* e41ca0a 2015-02-16 | Replace comments in draft and post tasks. [keighty]
...

git checkout -b <name of the local branch to create> upstream/<name of the remote branch>

git checkout - # checks out the last branch you were on
git checkout -p # checks out changes in chunks

I have already shared a few tools that have become so much a part of my tool box, and have developed such a muscle memory that my fingers will perform the tasks as I think them. Every time I pair with another developer I gain another shortcut! Thanks to Vince for those last two…

bird by bird

Sun, 01 Mar 2015 00:00:00 GMT

Virtue is its own reward, and so is writing. Anne Lamott’s Bird by Bird: Some Instructions on Writing and Life is full of beautiful prose, characters, and personal anecdotes, and while her advice is tuned to the creative writer, there are three useful lessons that can be applied to all writing.

write short assignments
write only what you can see through a 1-inch picture frame
write a terrible first draft

Blogging about what I am learning and reading forces me to think about what I experience. Almost every day I learn another 1-inch picture frame of programming, and writing it down helps me understand it better. My ugly first drafts usually consist of a question, a few links to resources, and some console output — not Pulitzer Prize winning prose, but an awesome personal resource and record of growth.

All you can give us is what life is about from your point of view. You are not going to be able to give us the plans to the submarine. Life is not a submarine. There are no plans.

uploading images from node and angular to s3

Tue, 24 Feb 2015 00:00:00 GMT

Amazon Web Services allow you to do everything, so it is hard to figure out how to do anything. I have a node/angular/mongo stack running on Heroku, and want to use Amazon to store images. I was delighted to stumble across Direct to S3 Uploads in Node.js, written by Will Webberly for the Heroku Dev Center blog. Following his example, I was able to upload images directly from the browser to S3, saving my app server a whole lot of work.

In the example, when a user selects a file for upload, the browser asks the node server for a temporary signed request. The server replies with a signed url, and the browser can send the data directly to Amazon. TADA — Almost. With a few exceptions. One of the prerequisites is that you know how to set up an S3 bucket and IAM user with the correct access controls.

Setting the S3 stage

When S3 receives a request it must verify that the requester has the proper permissions — both at the account level, as well as at the bucket level. With a seemingly infinite number of services available through Amazon, I found that I had to aggregate information from a few different sources.

Create a bucket
- See this Amazon tutorial for how to create a bucket.
Create a user
- See this Amazon tutorial for creating an IAM User.
- Be sure to record the generated Access Key ID and Secret Access Key. They will act as user name and password for accessing your bucket.
Grant the IAM User access to S3
- In the IAM User section of the AWS Management Console, select Attach Policy, and add AmazonS3FullAccess from the list
Grant the IAM User access to the bucket
- Use this Amazon tool to help you generate an Access Control Policy. The key pieces you need for the policy generator are the AWS Principle (the user you created), which you enter in the format arn:aws:iam::<your_account_number>:<IAM_user_name>, and the Resource (the bucket you created), which you enter in the format arn:aws:s3:::<your_bucket_name>/*.

{
  "Version":"2012-10-17",
  "Id": "Policy1234567890123",
  "Statement":[{
    "Sid":"Stmt123456789",
    "Effect":"Allow",
    "Principal": {
            "AWS": "arn:aws:iam::111122223333:specialUser"
    },
    "Action":[
      "s3:PutObject",
      "s3:DeleteObject",
      "s3:GetObject"
    ],
    "Resource":"arn:aws:s3:::examplebucket/*"
  }]
}

In case you get lost, here is one more invaluable Amazon tutorial.

Angular client side

Using an angular app on the client side is also a slight deviation from the Webberly tutorial. Angular does not support an ng-change binding on file input elements, but there is a workaround:

<input type="file" class="form-control btn" id="image" onchange="angular.element(this).scope().s3Upload(this)">

Additionally, instead of adding the upload function as plain javascript in your html template, add the s3Upload function onto the $scope in your controller.

$scope.s3Upload = function(){
  var status_elem = document.getElementById("status");
  var url_elem = document.getElementById("image_url");
  var preview_elem = document.getElementById("preview");
  var s3upload = new S3Upload({
      s3_object_name: showTitleUrl(),  // upload object with a custom name
      file_dom_selector: 'image',
      s3_sign_put_url: '/sign_s3',
      onProgress: function(percent, message) {
          status_elem.innerHTML = 'Upload progress: ' + percent + '% ' + message;
      },
      onFinishS3Put: function(public_url) {
          status_elem.innerHTML = 'Upload completed. Uploaded to: '+ public_url;
          url_elem.value = public_url;
          preview_elem.innerHTML = '<img src="'+ public_url +'" style="width:300px;" />';
      },
      onError: function(status) {
          status_elem.innerHTML = 'Upload error: ' + status;
      }
  });
};

Note the diff on line 6 — taking a peek into the S3Upload source, you can set a custom file name by passing a s3_object_name option. Otherwise, every object will be named default_name.

$scope.s3_upload = function(){
  ...
  var s3upload = new S3Upload({
    s3_object_name: showTitleUrl(),  // upload object with a custom name
    file_dom_selector: 'image',
    s3_sign_put_url: '/sign_s3',
  ...
}

function showTitleUrl() {
  var title = $scope.show.title.split(' ').join('_');
  var dateId = Date.now().toString();
  return dateId + title;
}

Add `image` to your schema

The last stumbling block I encountered was saving the image url into mongo. The image would upload and appear just fine, but wouldn’t persist the url as a property of the object. I had forgotten to add the image to my mongo Schema — the schema acts a bit like a whitelist for saving attributes through mongoose into mongo. Adding the imageUrl to the schema solved all my woes.

If you are trying to accomplish something similar, Heroku has articles on uploading directly to S3 using many different stacks.

everybody writes (and why you should blog)

Mon, 16 Feb 2015 00:00:00 GMT

In Everybody Writes¹, Ann Handley teaches content marketers how to generate “ridiculously good” content. Anyone who has recently enjoyed the pain of the job hunt will notice that much of the advice can be adapted to self-marketing. In my recent code school experience, developing an online presence was as important as learning web development, and since it is hard to market yourself as a web developer without developing anything for the web, a blog is a great place to start. You have picked a career where your ability to learn is one of your most marketable skills, as well as your ability to communicate — both of which you can showcase in your blog. You don’t have to worry about having something original to say: treat it as an opportunity to organize your thoughts and to teach a beginner what you know. Handley gives great advice for people learning to blog and how to generally optimize your online presence — both are great ways to market yourself as a fresh code-school alum.

Show up

Write early, write often. If something is new to you, it will likely be new to someone else as well. The more writing practice you get, the easier it becomes — “writing is a habit, not an art”, and you can’t help but improve in an art that you practice every day. Handley recommends starting with the ugly first draft² — make it ugly, don’t delete anything, don’t rephrase or reread. Just get it out of you and on to your screen, then walk away for a day. Maybe two. If you are really stuck on how to begin, you could start with “Dear diary”³. This helps you get in the mood to write, and keeps your tone conversational. Also, if you are writing to a non-technical person, it will keep your writing clear, and inspire you to give enough back story so that they aren’t lost.

Write a tight post

Decide on a goal

Is it to teach? To share? To critique? The goal for technical blogs is not usually to obtain a following — it is to share a bit of experience that will turn up when another frustrated developer puts keywords into a search engine. An online presence is important, but it should not be a blow by blow of how hard it is to learn to code. A handy formula for creating quality content is:

Utility * Inspiration * Empathy = Quality Content⁴

Make it useful, make it reader centric (even if that reader is you six months from now), and make it thoughtful.

Forget the 5 paragraph essay format from high school

I rarely read a long blog post from beginning to end, and the sweet spot for length is around 1500 words⁵. Make it easy for the reader to get the gist of your post by providing sub-headings, bold text, and code samples — these are strong visual elements that can help your reader anchor what they are learning.

Rewrite

Once you have written the ugly first draft with your goal in mind, rewrite it with the reader in mind — will they understand the context of what you are trying to get across? Assume the reader knows nothing, but isn’t stupid. Find data and examples that illustrate what you are trying to share (and make sure that you give credit where credit is due⁶).

Work on your grammar

After reframing, examine your sentence structure. Are you using a passive voice? Microsoft Word grammar check hates the passive voice, but I never knew what that meant. Try grammarly if you need help, and even if you think you don’t. Editing your own work is hard, because your brain will fill in all the things you meant to say but didn’t. Also, put the important words at the start of the sentence, just in case the reader dozes off half-way through.

About Me pages

Be intentional about the words that describe you in your About Me page⁷. Describe your accomplishments with examples, and if possible, try to mirror the language used by the companies you want to work for. Check out the keywords they use, and make sure to encorporate them in your headlines and personal summary.

About us pages aren’t really about the company; instead they focus on relaying who they are in relation to the visitor.

If your reader is an employer, let them know you are for hire — your posts should always be in the context of what you can do for the reader.

Showing up can be the hardest thing — finding something original to say, or fearing a public mistake is what keeps many people from writing, but blogging shows you are interested in, well, something. Being interesting means being interested in things, and wanting to share the things you find interesting is a natural follow through. Given the right mentorship, your technical abilities will grow, but getting in the door is step one. So, if you are looking to update your web presence, or create a web presence, this book is for you.

Handley, A. (2014). Everybody Writes : Your Go-To Guide to Creating Ridiculously Good Content. Wiley & Sons Canada, Limited, John.
↩
Tip 9: “Embrace the Ugly First Draft”, p. 41.
↩
Tip 14: “Start with Dear Mom…”, p. 54.
↩
“Introduction”, p. 7.
↩
Tip 60: “The Ideal Length for Blog Posts, Podcast, Facebook Posts, Tweets, and Other Marketing Content”, p. 184.
↩
Tip 57: “Seek Permission, Not Forgiveness”, p. 170.
↩
Tip 71: “Writing the About Us Page”, p 244.
↩

clean code

Sat, 07 Feb 2015 00:00:00 GMT

The book club I belong to has a two-fold mission: the first is to inspire through peer pressure to read (and finish) technical books, and the second is to improve our ability to write quality, maintainable code. We started with Practical Object Oriented Design in Ruby (Sandi Metz), and are now on to Refactoring (Martin Fowler and Kent Beck). Along the way, I found a copy of Clean Code by Robert C. Martin in our work library, and dove right in. These three books tout many of the the same principles but focus on different aspects of writing quality code: POODR teaches how to identify abstractions, and to design decoupled objects; Refactoring is a deep dive into the techniques and heuristics for eliminating common code smells; and Clean Code is the intersection between them. It is a style guide, a call to action, and a cautionary tale from the voices of experience. I had several aha moments while reading Clean Code, and here are a few:

Error handling

When I started to anticipate undefined method 'length' for nil:NilClass errors, I knew I was growing as a developer. Having traced the source of undefined is not a function enough times, I came to realize that I was not sufficiently controlling the messages passing between objects.

def find_a_user
  @user = User.where(last_name: "Bacon").first
end
=> :find_a_user

find_a_user
@user.last_name

NoMethodError: undefined method `last_name' for nil:NilClass
from (pry):10:in `__pry__'

This first pass seemed totally reasonable to me — find a particular user, and return it. Of course, what if the user doesn’t exist? What if the query timed out? The code on line 7 is reasonable (wherever it may live), but if @user is set to nil my app will blow up. The worst part is, it will blow up at quite a distance from where the actual problem is. A naive solution would be to check if @user is nil before calling #last_name on it (@user.last_name if @user), but this leads to unreadable, untrustworthy code. {“The problem isn’t that the code is missing a null check, but that it has too many.“} I have a poor contract with User if I have to check its validity every time I ask it a question. Martin suggests a more sustainable solution — never return nil. If you never return nil, you don’t have to catch nil.

If you are calling a [nil]-returning method from a third-party API, consider wrapping that method with a method that either throws an exception or returns a special case object.

def find_a_user
  if @user = User.where(last_name: "Bacon").first
    @user
  else
    raise StandardError.new "Could not find Mr. Bacon."
  end
end

find_a_user
=> StandardError: Could not find Mr. Bacon.

Now the error occurs where it belongs — with the retrieval of the user. Throwing an error when something goes wrong is the least you can do for the consumers of your method. You could also solidify your contract by guaranteeing that you return something they expect.

def interesting_things
  @interesting_things || []
end

interesting_things.length
=> 0

I have guaranteed that #interesting_things will always return either the collection I expect or a reasonable facscimile. It is a simple enough solution, but I definitely felt that message sink into place. Methods are objects and objects have API’s: messages in and messages out. Make those messages deliberate, because if “returning null from methods is bad, passing null into methods is worse.”

Protect Your Boundaries

Controlling the messages going in to and coming out of your objects is one place you need to protect your boundaries, but what about objects from 3rd party libraries or frameworks? There are consequences of relying too heavily on a library API, and in Chapter 9: Boundaries, James Grenning shares a few of them. One system he was working with collected information by passing around a Map collection, which is a standard library data structure in Java. In the next version of Java, new capabilities were added, and in order to use the new feature they had to find everywhere the object was used and make the change.

We’ve seen systems that are inhibited from using [the new feature] because of the sheer magnitude of changes needed to make up for the liberal use of Maps.

I recently had a similar experience when upgrading an application from Rails 3.0 to Rails 3.2, where there were drastic changes to how models record errors. Errors inherited from OrderedHash in Rails 3.0 and from ActiveModel in Rails 3.2 — very different object types. Every test that looked for errors called errors.on — now a deprecated method. Every place that looped over the collection with errors.each had to be changed to errors.messages.each, or the logic modified altogether. It is good practice to be distrustful of APIs that are likely to change, and while I don’t believe that the Errors object ever seemed unstable, it has certainly made me extra cautious of calling 3rd party framework methods directly.

Clean Code is not a “nice-to-have”

Technical debt is not a decision to be okay with untested, brittle code that is impossible to build on — it is a decision to pursue a design that may not be scalable, but it can still be clean. It is tempting to declare “Mission Accomplished” when you have reached the point of working code, but the job is really only half done:

It is not enough for code to work. Code that works is often badly broken. Programers who satisfy themselves with merely working code are behaving unprofessionally. They may fear that they don’t have time to improve the structure and design of their code, but I disagree. Nothing has a more profound and long term degrading effect upon a development project than bad code. Bad schedules can be redone, bad requirements can be redefined. Bad team dynamics can be repaired. But bad code rots and ferments, becoming an inexorable weight that drags the team down.

A story cannot be declared finished until the code is ammenable to change. In order to change code, it must be comprehensible and thoroughly tested. Compromising code quality to meet the demands of a schedule will only slow things down in the long run.

Clean Code is a must read for all professional software developers. Leaving clean code is not just a kindness to the next developer (which is likely to be you), but is essential for growing a healthy codebase. Of course, “to write clean code, you must first write dirty code and then clean it,” and I look forward to learning new techniques as we tackle Refactoring.

debugging techniques

Mon, 02 Feb 2015 00:00:00 GMT

I am working on upgrading an application from Rails 3.0 to Rails 3.2. While the numbers tell you it is a minor version change, there are definitely some major challenges. Digging through Rails and gem internals has forced me to add new strategies to my debugging toolbox, and here are a few of the essentials…

Zeus

Zeus is a gem that preloads your Rails app and keeps it loaded, making test driven development fast. Just type zeus start in one console window, and any of the listed commands in another console window.

$ zeus test test/models/user_test.rb
Run options: --seed 32298

# Running:

.

Finished in 0.045991s, 21.7434 runs/s, 21.7434 assertions/s.

1 runs, 1 assertions, 0 failures, 0 errors, 0 skips

Run a specific test in a file by passing a regex argument:

$ zeus test test/models/user_test.rb -n "/test_user_description_has_full_name/"
Run options: -n /test_user_description_has_full_name/ --seed 17927

# Running:

.

Finished in 0.044550s, 22.4467 runs/s, 22.4467 assertions/s.

1 runs, 1 assertions, 0 failures, 0 errors, 0 skips

Pry

The Pry gem allows you to set break points and open an interactive debug session at any place in your code using binding.pry. A great way to debug a failing test is to place a pry at the top of the test and check your setup:

require 'test_helper'

class UserTest < ActiveSupport::TestCase
  def test_user_description_has_full_name
    user = User.create(first_name: "hello", last_name: "kitty", email: "kitty@example.com")

    binding.pry

    assert_equal "hello kitty", user.full_name
  end

end

$ zeus test test/models/user_test.rb -n "/test_user_description_has_full_name/"

Run options: -n /test_user_description_has_full_name/ --seed 21519

# Running:

From: /Users/keighty/api_playground/test/models/user_test.rb @ line 7 UserTest#test_user_description_has_full_name:

     4: def test_user_description_has_full_name
     5:   user = User.create(first_name: "hello", last_name: "kitty", email: "kitty@example.com")
     6:
 =>  7:   binding.pry
     8:
     9:   assert_equal "hello kitty", user.full_name
    10: end

[1] pry(#<UserTest>)> user

=> #<User:0x007fa054180618
 id: 980190963,
 first_name: "hello",
 last_name: "kitty",
 email: "kitty@example.com",
 created_at: Sat, 07 Feb 2015 16:52:21 UTC +00:00,
 updated_at: Sat, 07 Feb 2015 16:52:21 UTC +00:00>

Your pry is a debug console with more than a few extra features. Here are my favourites:

ls object to see all the methods available on the object:

[3] pry(#<UserTest>)> ls user

ActiveRecord::Core#methods:
  <=>                 eql?                          hash          readonly!
  ==                  freeze                        init_with     readonly?
  connection_handler  frozen?                       inspect       set_transaction_state
  encode_with         has_transactional_callbacks?  pretty_print  slice
ActiveRecord::Persistence#methods:
  becomes     delete      increment!   toggle!           update_attributes
  becomes!    destroy!    new_record?  update            update_attributes!
  decrement   destroyed?  persisted?   update!           update_column
  decrement!  increment   toggle       update_attribute  update_columns
ActiveRecord::Scoping#methods:
  initialize_internals_callback  populate_with_current_scope_attributes
...

whereami 10 to show 10 lines above and below your breakpoint:

[5] pry(#<UserTest>)> whereami 10

From: /Users/keighty/api_playground/test/models/user_test.rb @ line 7 UserTest#test_user_description_has_full_name:

     1: require 'test_helper'
     2:
     3: class UserTest < ActiveSupport::TestCase
     4:   def test_user_description_has_full_name
     5:     user = User.create(first_name: "hello", last_name: "kitty", email: "kitty@example.com")
     6:
 =>  7:     binding.pry
     8:
     9:     assert_equal "hello kitty", user.full_name
    10:   end
    11:
    12: end

cd object.method to navigate the object stack as if it were a folder structure (and ls to view properites):

[1] pry(#<UserTest>)> cd user.full_name

[2] pry("hello kitty"):1> ls

Comparable#methods: <  <=  >  >=  between?
JSON::Ext::Generator::GeneratorMethods::String#methods:
  to_json_raw  to_json_raw_object  to_json_without_active_support_encoder
String#methods:
  %                  downcase        lstrip!           strip_heredoc
  *                  downcase!       match             sub
  +                  dump            mb_chars          sub!
  <<                 each_byte       next              succ
  <=>                each_char       next!             succ!
  ==                 each_codepoint  not_ascii_only?   sum
  ===                each_line       oct               swapcase
  =~                 empty?          ord               swapcase!

These are the features I use most often, but checkout the documentation for more.

Method Method Method

Method name collisions can happen in a sprawling legacy codebase, so determining the source is important for debugging. The Ruby language has anticipated this difficulty, and since methods are objects themselves, they created the Method module to access the method object. You can source, extract, unbind, and otherwise manipulate methods in a lot of ways:

[1] pry(#<UserTest>)> method = user.method(:full_name)
=> #<Method: User#full_name>

[2] pry(#<UserTest>)> method.owner
=> User(id: integer, first_name: string, last_name: string, email: string, created_at: datetime, updated_at: datetime)

[3] pry(#<UserTest>)> method.source_location
=> ["/Users/keighty/api_playground/app/models/user.rb", 2]

Logger

Soemtimes the Rails bug you are hunting turns up in the SQL behind your query. When you are in a pry console, the SQL is not displayed by default, but you can turn on query logging by creating an ActiveRecord::Base.logger object:

# before the logger
[1] pry(#<UserTest>)> User.find(980190963)

=> #<User:0x007fa0556f4fd0
 id: 980190963,
 first_name: "hello",
 last_name: "kitty",
 email: "kitty@example.com",
 created_at: Sat, 07 Feb 2015 16:52:21 UTC +00:00,
 updated_at: Sat, 07 Feb 2015 16:52:21 UTC +00:00>

[2] pry(#<UserTest>)> ActiveRecord::Base.logger = Logger.new STDOUT;

[3] pry(#<UserTest>)> User.find(980190963)

D, [2015-02-07T09:28:34.892017 #10468] DEBUG -- :
User Load (0.1ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = ? LIMIT 1  [["id", 980190963]]

=> #<User:0x007fa05c788c10
 id: 980190963,
 first_name: "hello",
 last_name: "kitty",
 email: "kitty@example.com",
 created_at: Sat, 07 Feb 2015 16:52:21 UTC +00:00,
 updated_at: Sat, 07 Feb 2015 16:52:21 UTC +00:00>

If you find that you are always creating a logger, add these lines to a .pryrc file in your home directory:

if defined?(Rails) && Rails.env
  require 'logger'
  ActiveRecord::Base.logger = Logger.new(STDOUT)
  ActiveRecord::Base.clear_active_connections!
end%

Globals

If you are bug hunting deep inside a gem or rails internals, and are trying to pry into a module that is loaded many times before it hits the buggy line of code, a simple binding.pry will have you stepping through an impossible amount of setup. Try setting a global variable to true right before the buggy line, and binding.pry on that condition:

➜  api_playground  zeus test /Users/keighty/api_playground/test/models/user_test.rb -n "/test_user_description_has_full_name/"
Run options: -n /test_user_description_has_full_name/ --seed 28811

# Running:


From: /Users/keighty/api_playground/app/models/user.rb @ line 5 User#full_name:

    2: def full_name
    3:   $bedug = true
    4:   "#{first_name} #{last_name}"
    5:   binding.pry if $bedug
    6: end

A global will maintain its state no matter how many times the code is loaded, so be sure to set the variable to false if you are stepping past the first break.

Caller

Calling caller at a pry prompt will output the current execution stack:

[1] pry(#<User>)> caller
=> ["/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:355:in `eval'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:355:in `evaluate_ruby'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:323:in `handle_line'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:243:in `block (2 levels) in eval'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:242:in `catch'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:242:in `block in eval'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:241:in `catch'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_instance.rb:241:in `eval'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:77:in `block in repl'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:67:in `loop'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:67:in `repl'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:38:in `block in start'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/input_lock.rb:61:in `call'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/input_lock.rb:61:in `__with_ownership'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/input_lock.rb:79:in `with_ownership'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:38:in `start'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/repl.rb:15:in `start'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/pry_class.rb:169:in `start'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/pry-0.10.1/lib/pry/core_extensions.rb:43:in `pry'",
 "/Users/keighty/api_playground/app/models/user.rb:5:in `full_name'",
 "/Users/keighty/api_playground/test/models/user_test.rb:6:in `test_user_description_has_full_name'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/minitest-5.5.1/lib/minitest/test.rb:108:in `block (3 levels) in run'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/minitest-5.5.1/lib/minitest/test.rb:206:in `capture_exceptions'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/minitest-5.5.1/lib/minitest/test.rb:105:in `block (2 levels) in run'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/minitest-5.5.1/lib/minitest/test.rb:258:in `time_it'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/minitest-5.5.1/lib/minitest/test.rb:104:in `block in run'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/minitest-5.5.1/lib/minitest.rb:327:in `on_signal'",
:

caller is a Kernel method. Pass a start line and a line limit to truncate the stack:

[11] pry(#<User>)> caller(20, 5)
=> ["/Users/keighty/api_playground/app/models/user.rb:5:in `full_name'",
 "/Users/keighty/api_playground/test/models/user_test.rb:6:in `test_user_description_has_full_name'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/minitest-5.5.1/lib/minitest/test.rb:108:in `block (3 levels) in run'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/minitest-5.5.1/lib/minitest/test.rb:206:in `capture_exceptions'",
 "/usr/local/var/rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/minitest-5.5.1/lib/minitest/test.rb:105:in `block (2 levels) in run'"]

The first 20 or so lines are usually from the pry session so they are easily ignored. I can see that the User.full_name method is being called from my test — a trivial case, but it comes in very handy when debugging a deeper stack.

Errors

When models have many association dependencies, satisfying the validations of many dependencies can be tricky without descriptive errors. In Rails 3.2, the error storage was changed from an OrderedHash to a first class ruby object in ActiveModel. While the interface has changed, its value remains:

[1] pry(main)> user = User.create
=> #<User:0x007fa054525318
 id: nil,
 first_name: nil,
 last_name: nil,
 email: nil,
 created_at: nil,
 updated_at: nil>

# Why does my object not have an id?

[2] pry(main)> user.errors
=> #<ActiveModel::Errors:0x007fa05451fda0
 @base=
  #<User:0x007fa054525318
   id: nil,
   first_name: nil,
   last_name: nil,
   email: nil,
   created_at: nil,
   updated_at: nil>,
 @messages={:first_name=>["can't be blank"], :last_name=>["can't be blank"]}>

[3] pry(main)> user.errors.full_messages
=> ["First name can't be blank", "Last name can't be blank"]

# oooooh... validations

scopes and single table inheritance

Thu, 22 Jan 2015 00:00:00 GMT

Single table inheritance (STI) is a way to emulate object-oriented inheritance in a relational database¹ by storing multiple object types in one table, distinguishable by a discriminator column such as type. Mixing levels of abstraction may make join operations easier, but it also makes other queries more complicated. In Rails 3.2, a query optimization was introduced that had some unintended consequences for STI.

A scope represents a narrowing of a database query, and a named scope is syntactic sugar for defining a class method at runtime.

class Shirt < ActiveRecord::Base
  scope :in_style, -> { where('purchase_date >= ?', Time.now - 2.months)}
end

class PoloShirt < Shirt
  scope :red, -> { where(color: 'red') }
  scope :blue, -> { where(color: 'blue') }
end

class SweatShirt < Shirt
  scope :logo, -> { where(logo: 'dragon' )}
end

PoloShirt.red.in_style

The scope :in_style is converted into a class method behind the scenes at runtime, and is defined on the singleton class where the scope was named, not on the caller. While this detail has no consequences for objects outside of an inheritance scheme, it means that when PoloShirt invokes the :in_style scope, the class method is declared on Shirt, not PoloShirt.

Here is where the history lesson begins. Scopes have evolved in Rails, and while they remain syntactic sugar for definition class methods, the details, method signature, and sql translation have differed dramatically.

In Rails 3.0, the scope method accepts a name, scope_options, and optional block. Scopes are directly translated into class methods behind the scenes, and the consequences of chaining scopes are the same as chaining queries, just nicer looking.

PoloShirt.red.in_style.to_sql
=> SELECT * from shirts WHERE color = 'red' where type = 'PoloShirt' where purchase_date >= 1417102745 where type IN ('PoloShirt', 'Sweatshirt')

Notice the implicit where clause in STI. The :red scope, declared on PoloShirt has where type = 'PoloShirt', and the :in_style scope, declared on Shirt, has where type IN ('PoloShirt', 'Sweatshirt'). The second where clause will have no impact on the query results, because they are already scoped to ‘PoloShirt’. It is this behaviour that evolves over time.

The first jump in scope evolution is in Rails 3.2, where the scope_options can include lambdas. Passing a lambda is a big advantage in that it allows the scope to be re-evaluated each time it is called. Unfortunately the implementation also remixes the query parameters of all chained scopes before evaluation:

In nested scopings, all previous parameters are overwritten by the innermost rule, with the exception of where, includes, and joins operations in Relation, which are merged.

PoloShirt.red.in_style.to_sql
=> SELECT * from shirts where color = 'red' where purchase_date >= 1417102745 where type IN ('PoloShirt', 'Sweatshirt')

The first where type = clause is merged with the last where type IN clause. The results of this query will no longer be scoped to PoloShirts, but will return all red.in_style Shirts of any type. Bad news for STI.

The good news is that this behaviour is fixed in Rails 4.0. The latest evolution of scopes no longer allow you to pass a non-callable object (like a hash), and all scopes are merged using AND.

PoloShirt.red.in_style.to_sql
=> SELECT * from shirts where color = 'red' where purchase_date >= 1417102745 where type = 'PoloShirt' AND type IN ('PoloShirt', 'Sweatshirt')

Our red.in_style query is once again scoped to the right level of the inheritance heirarchy. Named scopes are chainable and lazy-evaluated, making them a powerful query-building tool that can be difficult to troubleshoot — especially when they are mixed with single-table inheritance.

http://en.wikipedia.org/wiki/Single_Table_Inheritance
↩

Our healthcare system is a legacy application

Tue, 20 Jan 2015 00:00:00 GMT

One of my favourite blogs is Brain Pickings, written by Maria Popova, which is a collection of thoughts spanning art, science, psychology, and almost every other subject I read voraciously. Recently I stumbled upon this gem, Sherwin B. Nuland’s How We Die¹. It is a raw look at the biological processes that occur at the end of life, which resonated with me on many levels.

The biologist within was fascinated by Nuland’s ability to reduce the “ten thousand several doors for men to take their exits”² to the same basic process of poor oxygenation and nutrition of vital tissues resulting in organ failure. Whether the process is long or slow, instigated by heart disease, Alzheimer’s disease, AIDS, accident, or old age, the end must come when the heart stops nourishing the brain.

As a Clinical Professor of Surgery at Yale University, Nuland became intimate with death and its increasingly dehumanizing circumstances. Life’s final chapter has become invisible to most of us. It is likely to be carried out in an intensive care unit, the person reduced to data on a screen, and the advancing disease perceived as a failure of management rather than a natural and inevitable process. He shares the lessons he learned about providing true patient care rather than solving a biological riddle (sometimes the best patient care is no care at all).

Most importantly, he calls attention to the deplorable state of healthcare today, where the “provider” has usurped the role of the physician:

[Our] present system of in-hospital care… has simply accreted by the constant piling on or insinuation of patches of not necessarily compatible structure to an edifice already jerry-built of competing needs.

Sounds remarkably like a long-lived legacy application that is impossibly complex and difficult to work in. The healthcare system is in need of a great refactoring, which isn’t without precedent (read about Abraham Flexner’s report Medical Education in the United States and Canada that started the movement to raise “American medical education… to the highest [status] in the civilized world”³)

How We Die won the National Book Award, was a finalist for the Pulitzer Prize, and perched atop the New York Times’ bestseller list for more than thirty weeks. This book answers a question we have all asked at some point — what will it be like to die? A hope many of us share is to have a dignified death with little suffering for ourselves and our families, but reflecting on his many years and experiences with death, Nuland has found that

the greatest dignity to be found in death is the dignity of the life that preceded it. This is a form of hope we can all achieve, and it is the most abiding of all. Hope resides in the meaning of what our lives have been.

Nuland, Sherwin B. (2014-06-25). How We Die: Reflections on Life’s Final Chapter. Kindle Edition
↩
Webster, John. (1612) The Duchess of Malfi
↩
Flenxer, Abraham. (1910). Medical Education in the United States and Canada.
↩

objects vs data structures

Sun, 18 Jan 2015 00:00:00 GMT

“It is impossible to create an abstraction unknowingly or by accident,” says Sandi Metz in Practical Object Oriented Design in Ruby. An abstraction is a common, stable quality, such that you would find in a java interface. An interface is an idea that cannot be made concrete, but contains behaviour encoding similarities which objects might share. Even the definition of abstraction is abstract, and it wasn’t until I read Robert C. Martin’s chapter on Objects and Data Structures in Clean Code that I really started to understand them.

What is a data structure

A data structure is a class that exposes its data to the public. It should have no meaningful behaviour, and can be characterized by a set of attributes along with getters and setters.

class Point1
  attr_accessor :x, :y, :z
end

p = Point1.new
p.x = 1
p.y = 1
p.z = 1
p

In this example, a Point is defined as an object at some location on a coordinate grid of some kind. That sounds abstract enough: you can create a point, access all of the point’s data, and set its location along each axis independently. You have an instance of a point, but it is not an abstraction of a point — it is a data structure. {“Hiding implementation is not about putting a layer of functions between the variables. It is about abstractions.“} Adding an initializer that sets each variable, or a to_s method for getting a pretty output, would not make the Point1 class any more an abstraction of a position in space. To make an abstraction you can’t just use getters and setters — you have to think about how you are representing the data.

What is an object

An object hides its data behind abstractions. A Point is not defined by its [@x, @y, @z], but by its location in space. If space is a coordinate system, it could be 2- or 3-dimensional.

class Point2
  def initialize(x, y, z = nil)
    set_location(x, y, z)
  end

  def to_s
    "(#{[@x, @y, @z].compact.join(',')})"
  end
  alias_method :location, :to_s

  def set_location(x, y, z = nil)
    @x = x
    @y = y
    @z = z
    location
  end
  private :set_location

  alias_method :move_point, :set_location
  public :move_point

end

p = Point2.new(1, 1)
p.move_point(2, 3)
p.move_point(2, 3, 7)

A point cannot exist separate from it’s coordinates, and coordinates cannot exist or be altered without using the abstraction of moving in space.

“We do not want to expose the details of our data. Rather we want to express our data in abstract terms. This is not merely accomplished by using interfaces and/or getters and setters. Serious thought needs to be put into the best way to represent the data that an object contains.”

— Martin

This is why fat models are an anti-pattern in Rails. ActiveRecord makes an object out of a data structure, and adding behaviour beyond data validation makes the model a hybrid data object. Service Objects are abstractions of model behaviour (pun intended).

Now I know what an abstraction is, and I understand that simply extracting methods into interfaces doesn’t magically make an abstraction. As Metz says, “good design naturally progresses toward small independent objects that rely on abstractions.” An object is more than a data structure; it is an idea and it must be applied deliberately.

cleaning the campground

Sat, 10 Jan 2015 00:00:00 GMT

If you work in a code base with many other contributors, you may have learned the scouting philosophy to “leave the campground cleaner than you found it.” Sometimes the code you are working in can seem so entangled and complex that it is hard to know where to begin, but Robert C. Martin’s foundational book, *Clean Code*, tells us that very small changes can improve the readability and maintainability of a piece of code.

“It is not enough to write the code well. The code has to be kept clean over time.”

Code rot happens to every well-intentioned but long-lived code base, and it can happen either by neglect or by constant development. Neglected software will suffer a gradual deterioration in performance and responsiveness over time. Security vulnerabilities and incompatibilities aside, as new applications are built in newer and faster frameworks, your application will perform poorly in comparison. If your software is actively developed it will be modified, and as it is modified it’s complexity will increase. {“Add in more developers, each with different coding styles, and before long you will find your code to be impenetrably complex and impossible to modify.“}

There is hope for your legacy application, as long as you establish a culture of constant small improvements:

Change one variable name for the better
Break one large function into two or more smaller ones.
Eliminate one small bit of duplication
Clean up one composite if statement.

That last one is a bit like low-hanging fruit. Take this complex if statement in Rack::Deflater#call

def call
  ...

  # Skip compressing empty entity body responses and responses with
  # no-transform set.
  if Utils::STATUS_WITH_NO_ENTITY_BODY.include?(status) ||
      headers['Cache-Control'].to_s =~ /\bno-transform\b/ ||
     (headers['Content-Encoding'] && headers['Content-Encoding'] !~ /\bidentity\b/)
    return [status, headers, body]
  end
  ...

The comment attempts to explain what the code cannot: skip response compression under certain conditions. The intention could easily be described by extracting both the comment and the if statement to a method:

def call
  ...
  if skip_compression?
    return [status, headers, body]
  end
  ...
end

private

# Skip compressing empty entity body responses and responses with
# no-transform set.
def skip_compression?
  Utils::STATUS_WITH_NO_ENTITY_BODY.include?(status) ||
    headers['Cache-Control'].to_s =~ /\bno-transform\b/ ||
   (headers['Content-Encoding'] && headers['Content-Encoding'] !~ /\bidentity\b/)
end

While the if statement itself is not much better, at least #call is a little more readable. Fight against code rot while building new features in a legacy code base by making small improvements where you can.

#dup vs #clone in Ruby and rails

Sat, 10 Jan 2015 00:00:00 GMT

I was recently fixing a failing test and discovered that Ruby and Rails implement #clone and #dup in confusing and occasionally opposite ways.

In Rails `#clone` is a less complete copy of an object than `#dup`

Rails versions have flip-flopped on how to implement #clone and #dup, and there is ambiguity in how Rails defines “shallow”. In Rails 4.0, #clone is a shallow copy of an ActiveRecord object. “Shallow” in this context means that the clone shares attributes with the original:

Identical to Ruby’s clone method. This is a “shallow” copy. Be warned that your attributes are not copied. That means that modifying attributes of the clone will modify the original, since they will both point to the same attributes hash. If you need a copy of your attributes hash, please use the #dup method.

However, #dup is also described as a shallow copy. “Shallow” in this context means that while the dup does not share attributes with the original, it does share associations.

Duped objects have no id assigned and are treated as new records. Note that this is a “shallow” copy as it copies the object’s attributes only, not its associations. The extent of a “deep” copy is application specific and is therefore left to the application to implement according to its need.

`clone` vs `dup` in Rails:

pry> original = User.find(3)
  User Load (0.7ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = ? LIMIT 1  [["id", 3]]
=> #<User id: 3, first_name: "katie", last_name: "leonard", email: nil, created_at: "2015-01-10 17:37:00", updated_at: "2015-01-10 17:37:00">

pry> clone_copy = original.clone
=> #<User id: 3, first_name: "katie", last_name: "leonard", email: nil, created_at: "2015-01-10 17:37:00", updated_at: "2015-01-10 17:37:00">

pry> dup_copy = original.dup
=> #<User id: nil, first_name: "katie", last_name: "leonard", email: nil, created_at: nil, updated_at: nil>

Note that the clone_copy is an exact copy of the original (same user.id) and the dup_copy is a new record (user.id = nil). Any changes made to the clone_copy will be changed in the original, but any changes to the dup_copy attributes will remain isolated.

In Ruby `#clone` is a more complete copy of an object than `#dup`

With simple classes, clone() and dup() behave identically:

irb> class User
irb>   attr_accessor :first_name, :last_name, :email
irb>   def initialize(options={})
irb>     @first_name = options[:first_name]
irb>     @last_name  = options[:last_name]
irb>     @email      = options[:email]
irb>   end
irb> end
=> :initialize

irb> original = User.new(first_name: "katie", last_name: "leonard")
=> #<User:0x007fd7e98e0aa8 @first_name="katie", @last_name="leonard", @email=nil>

irb> cloned_copy = original.clone
=> #<User:0x007fd7e98c87c8 @first_name="katie", @last_name="leonard", @email=nil>

irb> dup_copy = original.dup
=> #<User:0x007fd7e98b24a0 @first_name="katie", @last_name="leonard", @email=nil>

irb> cloned_copy.first_name = "foo"
=> "foo"

irb> original.first_name
=> "katie"

irb> dup_copy.first_name
=> "katie"

irb> dup_copy.first_name = "bar"
=> "bar"

irb> original.first_name
=> "katie"

clone() and dup() function the same way!

clone() from the Ruby docs:

Produces a shallow copy of obj — the instance variables of obj are copied, but not the objects they reference. Copies the frozen and tainted state of obj. See also the discussion under Object#dup.

dup() from the Ruby docs looks suspiciously like the docs for clone():

Produces a shallow copy of obj — the instance variables of obj are copied, but not the objects they reference. dup copies the tainted state of obj. This method may have class-specific behavior. If so, that behavior will be documented under the #initialize_copy method of the class.

This deserves further clarification:

In general, clone and dup may have different semantics in descendant classes. While clone is used to duplicate an object, including its internal state, dup typically uses the class of the descendant object to create the new instance. When using dup any modules that the object has been extended with will not be copied.

To paraphrase, #dup will act like #clone, but without the original’s singleton class (ergo a “shallower” copy).

irb> class User
irb> attr_accessor :first_name, :last_name, :email
irb>   def initialize(options={})
irb>     @first_name = options[:first_name]
irb>     @last_name  = options[:last_name]
irb>     @email      = options[:email]
irb>   end
irb> end
=> :initialize

irb> module Crunchy
irb>   def bacon
irb>     "bacon"
irb>   end
irb> end
=> :bacon

irb> a = User.new(first_name: "katie", last_name: "leonard")
=> #<User:0x007fd7e8882490 @first_name="katie", @last_name="leonard", @email=nil>

irb> a.extend(Crunchy)
=> #<User:0x007fd7e8882490 @first_name="katie", @last_name="leonard", @email=nil>

irb> a.bacon
=> "bacon"

irb> b = a.clone
=> #<User:0x007fd7e8843060 @first_name="katie", @last_name="leonard", @email=nil>

irb> b.bacon
=> "bacon"

irb> c = a.dup
=> #<User:0x007fd7e98f05c0 @first_name="katie", @last_name="leonard", @email=nil>

irb> c.bacon
NoMethodError: undefined method `bacon' for #<User:0x007fd7e98f05c0>
        from (irb):101
        from /usr/local/var/rbenv/versions/2.1.5/bin/irb:11:in `<main>'

There are subtle differences between #clone and #dup in Ruby, and less subtle differences in Rails (depending on your version). Take care that the object you want is the object you get.

TDD and the senior software engineer

Sun, 04 Jan 2015 00:00:00 GMT

Many developers embrace test-driven development (TDD) as an ideal but rarely put it into practice. I often find myself writing code that works and then writing tests that exercise it to ensure that its behaviour doesn’t break when changes are made. Writing tests first is a great way to organize your approach to solving a problem, and in The Senior Software Engineer, David Copeland describes how TDD will help you grow your career as well.

A senior software engineer is efficient, and delivers a quality product in a timely fashion. “Effective problem solvers first spend time understanding the problem space and then design a plan”: documenting feature behaviour in a test suite allows you to plan your implementation. Code maintenance and modification are also more efficient when you have a good test suite. If the code has been built using TDD, its behaviour is already well tested. When someone needs to build on top of your feature, the tests will protect it and eliminate the worry of causing regressions (win-win).

Explicitly describing your plan in a test suite has another benefit that I had not considered before, and that is making you resilient to interruption. If your plan is not layed out on paper, you are maintaining it all in your head, and you have to hold on to a lot of context in your working memory. Every time you are interrupted, you have to dump things out of your working memory, and reloading the problem space takes time and energy. You won’t suffer interruptions well if reloading the context takes 15 minutes each time. You will become unresponsive as you block out the outside world, trying to maintain your context.

“TDD can help you structure your coding task so that it has natural breaking points. If you are interrupted while trying to fix a failing test, you know exactly where you can pick the task up again.” If you are interrupted after getting a test to pass you can easily read through your test suite to regain your context and pick up the next task.

Time and context management are essential skills for becoming a senior software engineer, and I am going to try using test driven development to become more interruptable and more productive.

octopress reloaded

Sat, 03 Jan 2015 00:00:00 GMT

I recently upgraded my home hardware from an Asus 10.1” netbook to a 15” MacBook Pro, and once I adjusted to being able to open more than one window at a time, I got down to the business of loading my files and projects on the new machine. Thanks to Dropbox and Github, most of that work happens either behind the scenes or as needed. Recreating the right environment for each project is still ongoing, and the first challenge I encountered was reloading my blog — started about two years ago using Octopress.

Octopress is an awesome framework built on jekyll. The learning curve for getting started with jekyll was a little steep (liquid layouts, yaml, not to mention design and UX were all new to me then) , and Octopress delivers on its promise to make it easy: just clone, bundle, and blog. Part of what makes Octopress so awesome is the deploy strategy: static pages are generated from yaml and markdown sources (saved in the source branch), and the pages are checked in to your github or heroku repository (as the master branch).

When you clone the repo, you get both branches, but you have to do a little setup to ensure that master is pulling from and pushing to the right place. Borrowing some code from the setup rake task:

$ git clone git@github.com:username/username.github.com.git my_blog
$ cd my_blog
$ git checkout source
$ mkdir _deploy
$ cd _deploy
$ git init
$ git remote add origin git@github.com:username/username.github.com.git
$ git pull origin master
$ cd ..
$ bundle install
$ bundle exec rake preview

Now I have no excuses to not blog more often… well, fewer excuses (thanks to Daniel Doubrovkine for pointing me in the right direction).

I briefly considered switching to a hosted blogging platform (tumblr or wordpress), so I conducted an informal survey (sample size taken from Digital Ocean’s 20 developers to follow) to see what choices leading developers were making, and it looks like a jekyll based blog is still the most popular. What is yours?

platform	users
jekyll	6
octopress	3
none	3
tumblr	1
hand-rolled	6
wordpress	1

the happiness hypothesis

Thu, 01 Jan 2015 00:00:00 GMT

“Nothing is miserable unless you think it so; and on the other hand, nothing brings happiness unless you are content with it.” —Buddha

In The Happiness Hypothesis: Finding Modern Truth in Ancient Wisdom, Jonathan Haidt provides an extensive overview of the state of positive psychology today.

He uses the analogy of a rider on an elephant to describe how the mind is divided into two parts: the conscious, reasoning part (the rider), and the unconscious mind that is actually driving our behaviour (the elephant). Stepping in to the New Year, the twitter-verse is alive with resolutions to eat less, exercise more, and spend less money — all laudable goals for the rider, but unless you can re-train the elephant to want those same things, you will find yourself in line for a 16oz hot chocolate with whipped cream by the end of the week.

On the happiness equation: H = S + C + V

The level of happiness that you actually experience (H) is determined by your biological set point (S) plus the conditions of your life (C) plus the voluntary activities (V) you do.

This has certainly been true in my life. I tend towards a positive outlook (as Haidt describes it, a winner of the ‘cortical lottery’), but that doesn’t mean I don’t experience ups and downs brought on by circumstances and choices. Changing your level of happiness can come from changing your circumstances, or by participating in rewarding acitivites. (Out of ideas? Try beginning ballet and wine tasting!)

On training the elephant

“The goal of meditation is to change automatic thought processes, thereby taming the elephant. And the proof of taming is the breaking of attachments… For Buddha, attachments are like a game of roulette in which someone else spins the wheel and the game is rigged: The more you play, the more you lose. The only way to win is to step away from the table. And the only way to step away, to make yourself not react to the ups and downs of life, is to meditate and tame the mind. Although you give up the pleasures of winning, you also give up the larger pains of losing.”

I like the idea of training the elephant through meditation, but disagree with the pursuit of non-attachment. Living life without any attachments means missing out on joy as well as pain. If life is an amuzement park, is it better to spend your time on the teacups or the rollercoaster? Isn’t it better to have loved and lost than never to have loved at all?

On your inner lawyer

[People] are skilled at finding reasons to support their gut feelings. The rider acts like a lawyer whom the elephant has hired to represent it in the court of public opinion… Studies of everyday reasoning show that… when people are given difficult questions to think about… they generally lean one way or the other right away, and then put a call in to reasoning to see whether support for that position is forthcoming.

I love the idea of our conscious mind acting as a lawyer defending our actions and decisions. We only look for supporting evidence, and any data that undermines our world-view must be discredited or its importance downplayed.

On reciprocation and gossip

A species equipped with vengeance and gratitude responses can support larger and more cooperative social groups because the payoff to cheaters is reduced by the costs they bear in making enemies… Tit for tat appears to be built into human nature as a set of moral emotions that make us want to return favor for favor, insult for insult…

Social groups cannot be formed without cooperation and a set of norms. Norms can be enforced through gossip:

Many species reciprocate, but only humans gossip, and much of what we gossip about is the value of other people as partners for reciprocal relationships… Gossip paired with reciprocity allow karma to work here on earth, not in the next life.

On writing your life story

The third level of personality is that of the “life story.” Human beings in every culture are fascinated by stories; we create them wherever we can.. It’s no different with our own lives. We can’t stop ourselves from creating what McAdams describes as an “evolving story that integrates a reconstructed past, perceived present, and anticipated future into a coherent and vitalizing life myth.” Although the lowest level of personality is mostly about the elephant, the life story is written primarily by the rider. You create your story in consciousness as you interpret your own behavior, and as you listen to other people’s thoughts about you.

Every time I relate a personal anecdote to a new listener I find that I am refining the narrative — using the same words and phrases, pausing at the same moments each time. My partner and I have developed a cadence when we tell people about our drive across Canada, or our trip to the Maritimes. We are building our life story through relatable moments.

On growing through adversity

Be sure you have done your best to answer these two questions: Why did this happen? What good might I derive from it?

Whatever doesn’t kill you doesn’t always make you stronger, but we can grow through adversity. Pain doesn’t always equal gain, but a root cause analysis (why did this happen?), and reframing to find something positive in the experience (lessons learned), is a great approach.

On the ideological differences in moral ethics:

The ethic of autonomy, the ethic of community, and the ethic of divinity.

Policial and ideological differences arise from approaching life from different moral perspectives. In the ethic of autonomy, the goal is to protect individuals from harm and interference. In the ethic of community the goal is to protect the integrity of groups, families, companies, or nations, and its virtues are obedience, and loyalty. In the ethic of divinity, the goal is to protect the sacred from the profane. They value living in a pure and holy way, free from pollutants such as lust, greed, and hatred.

Vital engagement (or flow)

Vital engagement does not reside in the person or in the environment; it exists in the relationship between the two.

Memories that encorporate more than one sensation are the strongest. Similarly, finding meaning and flow in an activity requires more than just a connection through commitment. There must be connections through a community, a shared history, and a shared experience. I didn’t come to love programming because it seemed like a rewarding career path. I built a script that was useful to a coworker, drew excitment from other programmers, and began peeling back the layers of abstraction, enjoying the ‘A-ha!’ when understanding occurred.

On happiness and the meaning of life

Happiness is not something that you can find, acquire, or achieve directly. You have to get the conidtions right and then wait.

This is the first book I have read that really tried to approach the meaning of life, and my inner elephant has decided that Haidt is on the right track.

method lookup in ruby

Sun, 28 Dec 2014 00:00:00 GMT

I have once more been working my way through Sandi Metz’ Practical Object-oriented Design in Ruby (POODR) for a few weeks, and developed a mental block about method lookup.

In Chapter 6 (Acquiring behaviour through inheritance), she describes how to extract a superclass from a group of related classes that share some behaviour by pulling methods up the inheritance chain instead of driving specializations down. This approach ensures a clean abstraction, leaving no specialized behaviour in the superclass. Where I got stuck was in the call chain that enforces the common interface:

class Foo
  def initialize
    baz
  end

  def baz
    raise NotImplementedError.new "You can't call #baz here!"
  end
end

class Bar < Foo
  def initialize
    super
  end

  def baz
    puts "Hello, baz"
  end
end

> x = Bar.new
"Hello, baz"
=> #<Bar:0x007fc4f903ab90>

I could not understand why, the NotImplementedError wasn’t raised when a new Bar was created. This is how I imagined the call stack should work:

Bar.new calls Bar#initialize
Bar calls super (Foo#initialize)
Foo calls baz (Foo#baz)
raises NotImplementedError

This is how the call stack actually works

x = Bar.new calls Bar#initialize
x receives a call to super (Foo#initalize)
x receives a call to baz (Bar#baz)
says “Hello, baz”

The call never comes from the context of Foo — the receiver is always x, the instance of Bar. Classical inheritance dictates that an object can only look to itself or further up the inheritance chain for valid method definitions, not down. x is reaching up to call super but checks itself for an answer to the question #baz. Foo#baz would only ever be called if we made a new Foo directly:

> y = Foo.new
NotImplementedError: You can't call #baz here!
        from (irb):7:in `baz'
        from (irb):3:in `initialize'
        from (irb):27:in `new'
        from (irb):27
        from /usr/local/var/rbenv/versions/2.1.5/bin/irb:11:in `<main>'

How ruby resolves a message (classical inheritance)

The search for a method begins in the class of the receiving object — in the example above, Bar is always the receiving object. If the class does not implement the message, the search proceeds up the superclass chain. If none of the superclasses contain the method definition, ruby makes a second attempt to resolve the message by sending method_missing(:method_name) to the original object. The search restarts from the bottom, but this time for a method_missing handler rather than :method_name.

Occasionally refreshing the ruby basics is super-rewarding.

compilers -- an intro

Sat, 08 Nov 2014 00:00:00 GMT

Last year, I was wrapping up a stint at code school, and I picked up the challenge of building a programming language. It was a very large stretch-goal, and I didn’t complete the project, but I learned a lot about how languages are designed and built. Recently, my mentor highlighted a Compilers course offered through Stanford Online, and suggested that it may be time to level-up my understanding of languages once again. Since I did not reach the compiler phase for eh?, perhaps this is an opportunity to do so.

Review of compilers

A compiler is different from an interpreter. An interpreter takes both the data and the program and produces output. A compiler takes only the program and produces an executable. Fun fact: I learned just recently that javascript is actually a “compiled” language — the javascript engine uses just-in-time compilation as it is executed.

The grandfather of all compiled languages was FORTRAN, which was developed to translate formulas into a machine-readable language. Like many software projects, the initial completion date was wildly optimistic — it took 7 years longer than their original estimate — but they developed a 5-stage pattern of processing that all later compilers would follow.

1. Lexical analysis

This first step is about word and symbol recognition: breaking a stream of characters into recognizable chunks. For example a method definition:

# Ruby
def my_method; end

// Java
public static void main() { }

# eh
can curl; eh?

Recognizing white space, punctuation, and words is the first step in translating characters into actions that can be performed. The words that are recognized are usually converted into “tokens”, for passing in to the next stage of processing, which is why this step is occasionally called Tokenizing.

2. Parsing

After the character stream has been tokenized, doing pattern recognition on the sequence of tokens is called parsing. Returning briefly to the example of a method definition, def my_method; end, the lexer would recognize def as a key word and my_method as a variable, but they are grouped together by the parser as the beginning of a method block.

3. Semantic analysis

Compilers can only do a very limited amount of semantic analysis, usually limited to catching inconsistencies (such as redeclaring a variable) and ambiguities (such as type mismatches). “A panda bear walks into a bar and eats, shoots and leaves”, can be tokenized into component parts (verbs, nouns, articles), as well as into phrases (subject, predicate), but determining the exact meaning of the predicate requires deeper understanding of the subject. Machines do not perform well with ambiguous grammar, and developing an unambiguous language syntax is a difficult task.

4. Optimization

Optimization is where most modern language compilers spend the majority of their resources, modifying programs so that they reduce the demand for resources like memory allocation, or garbage collection. For example, the java compiler will “unroll” all the for loops so that they run more efficiently. Another strategy is peephole optimization, where the compiler checks out adjacent code to see if any assignments could be reduced or replaced by simple math.

x = y * 2 might be reduced to x = y + y.

z = 0; x = y * z might be reduced to x = 0, but there is a gotcha hidden there — y * 0 = 0 is only true for integers. It results in NaN for floating point numbers.

5. Code generation

Code generation is the final translation of the program into an executable. The written language has been cut into tokens and parsed into phrases. It has been examined for ambiguities and inconsistencies, and optimized wherever possible. The instructions for the program are arranged neatly and must now be evaluated and scheduled. The final product is an executable program that can be run against any data of interest.

This 5 stage pattern is followed by almost all compilers. Where they vary is in their proportions. FORTRAN spent a balanced amount of time in each of the 5 phases, while modern compilers have a much longer optimization phase than lexing or parsing.

how to solve it

Thu, 06 Nov 2014 00:00:00 GMT

4 phases:

Understand the problem - see clearly what is required
Find connections and make a plan
Carry out the plan
Look back at the completed solution - review and discuss

“To devise a plan, to conceive of the idea of a solution is not easy. It takes so much to succeed: formerly acquired knowledge, good mental habits, concentration on the purpose, and one more thing: good luck. To carry out the plan is much easier; what we need is mainly patience.”

“…by reconsidering and examining the result and the path that led to it, they could consolidate their knowledge and develop their ability to solve problems.”

“There remains always something to do; with sufficient study and penetration, we could improve any solution, and, in any case, we can always improve our understanding of the solution.”

Where do I start?

Start with the problem statement

What can I do?

Isolate the principle parts of the problem

A problem to prove has a hypothesis and a conclusion A problem to find has data, conditions, and an unknown

If a problem is very complex, you may distinguish great and small steps. Consider the great ones first and get down to the smaller ones after.

Looking back

start with the solution consider the solution from various sides scrutinize the method that led you to the solution

meteor-managed ids won't play nicely with others

Sat, 18 Oct 2014 00:00:00 GMT

I am working with a data dump from a meteor project, and while retrieving and displaying the existing collection was no problem at all, I was stuck on saving new documents to mongo.

My schema was pretty straightforward and taken directly from an existing document:

var postSchema = mongoose.Schema({
  "title" : String,
  "company" : String,
  "author" : String,
  "music" : String,
  "choreographer" : String,
  "showDate" : String,
  "image": String,
  "userId" : String,
  "postAuthor" : String,
  "submitted" : Number,
  "commentsCount" : Number,
  "_id":  String
});

The problem was that last field: _id. If I included it in the schema then I would have the objectId available on the front end but would get an error trying to insert a new document: [Error: document must have an _id before saving]. If I removed _id from the schema I could save documents just fine but could not pass through the objectIds of already existing documents. I have learned that mongo is very flexible when it comes to assigning ids before insertion into the database.

Mongo objects created through a meteor application are given meteor-friendly objectIds, the uniqueness of which is monitored and maintained by a meteor wrapper class. The design decision to go with ids as strings seems to be motivated by meteor’s latency compensation feature — creating documents on the client-side and then later syncing them with the server requires that they are assigned objectIds before they ever reach mongo. Mongo will accept a manually entered id, trusting that there is an entity somewhere that is managing uniqueness.

Here was my problem: the documents created by meteor had String ids assigned by meteor. If I included _id in my schema, mongo would assume that I was managing id assignment manually, and wouldn’t save the object until I did. If I did not include _id in my schema, mongo would assume it was responsible, and would assign a unique id accordingly.

To make the documents created by meteor play nice with the new stack, I had to update each document’s _id with one created by mongo. Unfortunately, mongo won’t allow modification of _id directly, so the only choice was to recreate and insert each entry, then delete the original:

Retrieve a document: doc = db.posts.findOne()
Save the title: title = doc.title
Save the document’s string id: id = doc._id
Reassign the document id: doc._id = ObjectId()
Save the updated document: db.posts.insert(doc)
Remove the original document: db.posts.remove({_id: id})
Verify that the post exists with the new id: db.posts.findOne({"title": title})

> doc = db.posts.findOne()
{
  "title" : "La Cage aux Folles",
  "company" : "Pixie Dust Productions",
  "author" : "Harvey Fierstein",
  "music" : "Jerry Herman",
  "showDate" : ISODate("2014-09-21T07:00:00Z"),
  "dateSubmitted" : 1411346259065,
  "_id" : "EjzopWQa3mw5LB979"
}

> title = doc.title
La Cage aux Folles

> id = doc._id
EjzopWQa3mw5LB979

> doc._id = ObjectId()
ObjectId("5442af27cc37fe6f03648fdd")

> db.posts.insert(doc)

> db.posts.remove({_id: id})

> db.posts.findOne({"title": title})
{
  "_id" : ObjectId("5442af27cc37fe6f03648fdd"),
  "title" : "La Cage aux Folles",
  "company" : "Pixie Dust Productions",
  "author" : "Harvey Fierstein",
  "music" : "Jerry Herman",
  "showDate" : ISODate("2014-09-21T07:00:00Z"),
  "dateSubmitted" : 1411346259065,
  "commentsCount" : 1
}
>

Here is a handy script for changing the entire collection at once:

db.posts.find().forEach(function(doc){ var doc = db.posts.findOne(); var title = doc.title; var id = doc._id; doc._id = ObjectId(); db.posts.insert(doc); db.posts.remove({_id: id}); })

Awesome.

Sort all the things with mongo and node.js

Thu, 09 Oct 2014 00:00:00 GMT

Sorting shouldn’t be difficult, but through my experience learning meteor I discovered that if mongo likes sort commands one way:

db.posts.find().sort({submitted: -1})

meteor would like them a different way:

Posts.find({}, {sort: {submitted : -1}}

I am adapting one of my meteor projects to use mongo and node.js, connected through mongoose, and fitting in the callback required for node sent me to the google. After a few stackoverflow searches I turned up a several options:

Post.find({}, {sort:[['submitted',-1]]}, function(err, doc) {
  response.send(doc);
});

This option returned a object containing only the entry ids, not the full documents:

0: {_id:2tcte4Srd8QMSqrKt}
1: {_id:5c3FdWphiYiMQzukw}
2: {_id:62vAt5ewucKm542DH}
3: {_id:AWNYsxBgFnv9Z4NSA}
...

Another option suggested that converting the result to an Array would complete the query:

Post.find({},{sort: [['submitted',-1]]}).toArray(function(e, results){
  response.send(results);
});

This option resulted in TypeError: Object #<Query> has no method 'toArray'.

Rather than continuing to search for a snippet to steal, I turned to the source, and it turns out that querying mongo from mongoose is simpler than either of these strategies.

Post.find()
  .sort("-submitted")
  .exec(function(err, doc) {
    response.send(doc);
  });

Success! Why? Because while mongo itself returns a cursor object which can be transformed into documents using toArray(), or fetch() in the case of meteor, mongoose returns a Query object, which will return the full documents once it is passed a callback.

A Query object can be built using method chaining — each method (find | where | limit | select | sort) returns a new Query object, which allows you to build in stages:

Post
.find({ title: /the/ }) // query
.where('cost').gt(17).lt(66)  // query
.where('location').in(['Schnitz', 'Armory'])  // query
.limit(10)  // query
.sort('-ticketDate')  // query
.select('title company')  // query
.exec(callback);  // EXECUTE THE QUERY

Another fun bit of javascript/mongoose magic, is that you can indicate sorting in descending order by prefacing the sort string with a ”-” :

Post
.find({ title: /the/ }) // query
.sort('-ticketDate')  // DESC!
.exec(callback);  // EXECUTE THE QUERY

Awesome.

Resources

mongo docs

stackoverflow

giving the cold shoulder with git

Mon, 08 Sep 2014 00:00:00 GMT

Some files become unnecessary to track using version control, but still belong in the repository. Compiled assets, for example may require a placeholder for a functional deploy, but don’t need to be checked in every time a change is made.

Adding the file to your gitignore is not enough to stop tracking file changes. You must also clear the file from your git cache:

$ git rm --cached [filename]
$ git add .
$ git commit -m "I really mean to ignore this file."

To untrack everything that has been added to the .gitignore:

$ git rm -r --cached .
$ git commit -a -m "I mean it!"

Resources

Here’s where I found this gem, and here are the docs where I verified it.

gaining closure through modules

Sat, 06 Sep 2014 00:00:00 GMT

If you have more than one javascript running on a page, you have likely experienced the hell of trying to debug a problem caused by naming collisions. Developing the scripts in isolation, it makes sense to assign widgetA.name dynamically using jQuery, but then why does the title of widgetGraphB disappear? Avoid the pain of unintended consequences by namespacing your scripts.

Closures occur when you pass functions to variables to call at a later date. The javascript engine does all the work of compiling the script and loading the contents into the memory stack, but until the function is called the stack just sits there, sealed like a time capsule.

function timeCapsule() {
  var year = 1999;
  function party() {
    console.log("We're gonna party like it's " + year);
  }

  return party;
}

var year = 2099;
var tonight = timeCapsule();

tonight(); // We're gonna party like it's 1999

The party() function has a closure over timeCapsule(), which means all the variables and functions within timeCapsule remain unmolested by any assignments made outside of their scope.

Modules use closure to encapsulate scope.

Breaking your monolithic javascript into modules is a great way to separate concerns and create closures.

function myModuleFunction() { /* Do work1 */ }
function mySecondModuleFuntion() { /* Do work2 */ }

exports.myModuleFunction = myModuleFunction;
exports.mySecondModuleFunction = mySecondModuleFunction;

var myModule = require("myModule");

myModule.myModuleFunction(); // performs work1
myModule.mySecondModuleFunction(); // performs work2

Using modules will prevent naming conflicts on functions and variables:

var module1 = require("myModuleName");

myModuleFunction() {
  console.log("All work and no play makes module1 a dull script;");
}

module1.myModuleFunction(); // performs work1
myModuleFunction(); // outputs snarky phrase

Voila — no conflicts.

backup meteor? yes we can

Sun, 24 Aug 2014 00:00:00 GMT

Retrieving a data dump from your meteor production database and copying it to local development is a three-step, time sensitive process.

1. Retrieve your production url

Passing a -url flag to the meteor mongo command will return a long string of goodies:

$ meteor mongo --url my_app.meteor.com

mongodb://client-12345678:9abcdef1-234-5678-9abc-def123456789@singularsensation-db-a3.meteor.io:12345/my_app_meteor_net

This string contains all the information you require to access your production data:

your client id (from // to :)
a server password (from : to @)
a server name and port number (from @ to /)
the datastore identification (from / to the end)

mongodb://<CLIENT_ID>:<PASSWORD_HASH>@<SERVER_NAME:PORT>/<YOUR_DATA_STORE>

Retrieving your data is time-sensitive because the password hash will expire after 60 seconds.

2. Retrieve your data using mongodump

$ mongodump -u CLIENT_ID -h SERVER_NAME:PORT -d YOUR_DATA_STORE -p PASSWORD_HASH

connected to: <SERVER_NAME:PORT>
Sun Aug 24 10:50:32.342 DATABASE: YOUR_DATA_STORE  to   dump/YOUR_DATA_STORE
Sun Aug 24 10:50:32.989   YOUR_DATA_STORE.system.indexes to dump/YOUR_DATA_STORE/system.indexes.bson
Sun Aug 24 10:50:34.368      14 objects
Sun Aug 24 10:50:34.369   YOUR_DATA_STORE.system.users to dump/YOUR_DATA_STORE/system.users.bson
Sun Aug 24 10:50:34.559      2 objects
... and on and on

Mongodb copies your data from production into a new folder called “dump” in your present working directory.

3. Use mongorestore to copy the data from the dump into your local datastore

$ mongorestore --host 127.0.0.1 --port 3001 --db meteor --drop dump/YOUR_DATA_STORE/

connected to: 127.0.0.1:3001
Sun Aug 24 11:36:54.693 dump/YOUR_DATA_STORE/users.bson
Sun Aug 24 11:36:54.693   going into namespace [meteor.users]
Sun Aug 24 11:36:54.693    dropping
2 objects found
Sun Aug 24 11:36:55.834   Creating index: { name: "_id_", key: { _id: 1 }, ns: "meteor.users" }
Sun Aug 24 11:36:55.993   Creating index: { name: "username_1", key: { username: 1 }, unique: true, ns: "meteor.users", sparse: 1 }
... and on and on

Awesome.

Resources: stackoverflow

formatting numbers the easy way

Sat, 23 Aug 2014 00:00:00 GMT

D3 has an awesome number formatting function that I discovered only after hacking my own. My first approach was to use string and array manipulation to convert 1234567890 to 1,234,567,890:

function numberSmoothing(value) {
  // split the number into a character array and reverse it
  reversedNumber = (value).toString().split('').reverse();

  // ["0", "9", "8", "7", "6", "5", "4", "3", "2", "1"]

  // push the numbers onto another array in groups of 3 and add a comma
  chunkedNumber = [];
  while(reversedNumber.length > 3) {
    var numberChunk = reversedNumber.splice(3);
    chunkedNumber.push(reversedNumber.join('') + ',');
    reversedNumber = numberChunk;
  }

  // push the remaining digits onto the array
  chunkedNumber.push(reversedNumber.join(''));

  // ["098,", "765,", "432,", "1"]

  // reverse each of the strings in the array and rejoin them
  for(var j = 0; j < chunkedNumber.length; j++){
    chunkedNumber[j] = chunkedNumber[j].split('').reverse().join('');
  }

  // [",890", ",567", ",234", "1"]

  return chunkedNumber.reverse().join('');
}

> numberSmoothing(1234567890)
"1,234,567,890"

While it was a fun mental exercise to implement number formatting, I stumbled across this handy d3 function after I was already done:

> var format = d3.format(",");
> format(1234567890);
"1,234,567,890"

As it turns out, any way one might want to format a number has already been implemented in d3. It is similar to the mini-formatting language from python:

[[fill]align][sign][symbol][0][width][,][.precision][type]

// add commas for the thousands separator
> var format = d3.format(',')
format(1234567890)
"1,234,567,890"

// add a sign for positive and negative numbers
> var format = d3.format('+,')
> format(1234567890)
"+1,234,567,890"

> format(-1234567890)
"-1,234,567,890"

// add a currency sign
> var format = d3.format('+$,')
format(1234567890)
"+$1,234,567,890"

// as a percentage
> var format = d3.format('p')
format(1/6)
"16.666666666666664%"

// as a rounded percentage
> var format = d3.format('%')
> format(1/6)
"17%"

// as a percent rounded to a single sig fig
> var format = d3.format('.1%')
> format(1/6)
"16.7%"

// as a rounded value
> var format = d3.format('.2r')
> format(123.456)
"120"

>format(123456789.66)
"120000000"

// as a rounded, comma separated value
> var format = d3.format(',.2r')
> format(123456789.66)
"120,000,000"

Awesome.

javascript getters and setters

Sat, 09 Aug 2014 00:00:00 GMT

Getters and setters make sense in java, and I just learned that there is an equivalent pattern in javascript:

> var favouriteBook = {
  _title: "Catch-22",
  get title() {
    console.log("Getting title", this._title);
  },
  set title(value) {
    this._title = value;
    console.log("Setting title", this._title);
  }
}
> favouriteBook.title
 Getting title Catch-22

> favouriteBook.title = "Slaughterhouse 5"
 Setting title Slaughterhouse 5

> favouriteBook.title
 Getting title Slaughterhouse 5

The get and set key words are reserved for accessing or mutating a data property (instance variable). Of course, one can get and set data properties directly:

> favouriteBook.title
"Catch-22"

> favouriteBook.title = "Slaughterhouse 5"
> favouriteBook.title
"Slaughterhouse 5"

The advantage of using a getter and setter is that if a setter is defined without a getter, one can change the value of the data property, but can never read it. If a getter is defined without a setter, then the variable can be read but not be changed:

> var favouriteBook = {
  _title: "Catch-22",
  get title() {
    console.log("Getting title", this._title);
  }
}

> favouriteBook.title
Getting title Catch-22

> favouriteBook.title = "Slaughterhouse 5"

> favouriteBook.title
Getting title Catch-22

The setter function will fail silently, which makes troubleshooting difficult but ensures that the property remains undisturbed.

Resources

The principles of object-oriented javascript

constructors and prototypes

Thu, 07 Aug 2014 00:00:00 GMT

Object-oriented programming (OOP) can be thought of as describing the properties and behaviour of a noun in a blueprint, which is used to create instances of an object. The blueprint is defined as a Constructor — a function that identifies a class of objects and begins with a capital letter. New objects are created by calling a Constructor with the keyword new.

> function Book(title, author) {
    this.title = title;
    this.author = author;
    this.listing = function() {
      console.log([this.title, "by", this.author].join(' '));
    }
  }

> var book1 = new Book("Catch-22", "Joseph Heller")

> book1.title
"Catch-22"

> book1.author
"Joseph Heller"

> book1.listing()
Catch-22 by Joseph Heller

The Book constructor specifies two properties, title and author, and one method, listing. Defining a method in a constructor ensures that all instances of the Object have the same method, but it also results in duplication of the method in the memory stack for every instance of the Object. If I made 1000 instances of Book, there would be 1000 instances of the listing method, even though it is only defined once, and is the same for every object.

Memory stack of objects with instance methods:

Enter Prototypes.

A Prototype is a property that is shared by all instances of an object. When the constructor for an object is called for the first time, all the object’s properties are loaded onto the memory stack as well as the object’s Prototype. All subsequent instances of that object include a pointer to the same Prototype, and they pass in a pointer to themselves as this whenever they call a method from the Prototype.

function Book(title, author) {
  this.title = title;
  this.author = author;
}

Book.prototype = {
  listing: function() {
    console.log([this.title, "by", this.author].join(' '));
  }
}

> var book1 = new Book("Catch-22", "Joseph Heller")
> var book2 = new Book("Slaughterhouse 5", "Kurt Vonnegut")

> book1.listing()
Catch-22 by Joseph Heller
> book2.listing()
Slaughterhouse 5 by Kurt Vonnegut

> book1.hasOwnProperty("listing")
false
> 'listing' in book1
true

Memory stack of object with prototypes:

Reduced memory consumption is one advantage of keeping a reference to a prototype instead of a complete copy. Another advantage is that if additions are made to the Prototype, the changes will be extended to all instances.

Awesome.

'in' operator avoids false falsies

Tue, 05 Aug 2014 00:00:00 GMT

I have encountered more than my share of javascript errors because I have made assumptions about the object I am working with, such as assuming that an object contains a property, and that I can call methods on that property:

> myObject.data[0]
TypeError: Cannot read property '0' of undefined

This line contains four assumptions, and therefore four places it can fail:

myObject exists
myObject contains a property called data
data is an array
data is non-empty

Presuming that if myObject does not exist means you have bigger problems, it is easy to fall into the trap of checking for the existence of a property by gating a conditional with it:

if(myObject.data) {
  //do something important with the data
}

This technique will work in development, but the voice of experience tells us that it will not work in production. The reason it will not work is that javascript has more falsy values than most languages. Falsy values include the number zero, empty strings, null, undefined, NaN, and of course, the boolean false. So, if it is possible myObject.data to contain derived information, you have signed up to troubleshoot some unexpected behaviour.

Fortunately, javascript has provided a failsafe way to check the existence of a property, regardless of the data it contains. The in operator checks if the key exists in the object’s hash table, and doesn’t care if it is a prototype or singleton property.

> var myObject = {}
undefined
> 'data' in myObject
false

> myObject.data = ''
""
> 'data' in myObject
true
> myObject.data == true
false

('data' in myObject) returns false if the property is undefined. If the property is defined, it returns true even if data contains a falsy value.

Resources

The principles of object-oriented javascript

stringify cyclic structures with censors

Wed, 23 Jul 2014 00:00:00 GMT

Troubleshooting a d3 data visualization sometimes requires visual inspection of the underlying JSON data structure. You can convert linear data structures to a JSON string using JSON.stringify:

> x = {}
> JSON.stringify(x)
"{}"
> x.y = ["1", "2", "3"]
> JSON.stringify(x)
"{"y":["1","2","3"]}"
> x.z = {a: ["b", "c"]}
> JSON.stringify(x)
"{"y":["1","2","3"],"z":{"a":["b","c"]}}"

Data structures that contain both upstream and downstream links between objects are cyclic structures. From the docs:

JSON does not support cyclic structures. Attempting to convert such an object into JSON format will result in a TypeError exception.

> x = {}
  y = {}
  x.children = [y]
  y.parent = x
Object {children: [Object]}

> JSON.stringify(x)
TypeError: Converting circular structure to JSON

How can I look at a tree, or double-linked list data structure that is rife with circular links? JSON.stringify() takes a second argument called a censor in the docs. The censor is the programmatic court of appeal for whenever a cyclic structure is encountered:

function censor(key, value) {
  if (typeof value === "object") {
    return undefined;
  }
  return value;
}

Of course, this approach will eliminate most of the tree structure by removing all objects from the JSON. Certainly we can return something more creative than undefined…

function censor(key, value) {
  if (value && typeof value === "object" && value.parent) {
    value.parent = value.parent.name;
  }
  return value;
}

In this censor, we replace the parent object with the parent’s name, and JSON.stringify has no complaints:

> function censor(key, value) {
  if (value && typeof value === "object" && value.parent) {
    value.parent = value.parent.name;
  }
  return value;
}

> x = {name: "Theodore"}
Object {name: "Theodore"}
> y = {name: "Alvin"}
Object {name: "Alvin"}
> x.children = [y]
[Object]
> y.parent = x
> Object {name: "Theodore", children: Array[1]}

> JSON.stringify(x)
TypeError: Converting circular structure to JSON

> JSON.stringify(x, censor)
"{"name":"Theodore","children":[{"name":"Alvin","parent":"Theodore"}]}"

Awesome.

jQuery-UI and meteor

Mon, 16 Jun 2014 00:00:00 GMT

I have been working on Virtual Playbill for the last few weeks using meteor, bootstrap, and a number of other UI packages. I wanted to use a datepicker as a form field, but getting jQuery-ui working was not a straightforward package download as many other features are. There are currently 5 packages available for installation on Atmosphere (the meteor package manager), and I attempted several of them before finding the combination that worked:

1. Install the jQuery-ui package from Atmosphere

mrt add jquery-ui

2. Download jQuery-ui

For me, the meteorite package did not properly install the css or images I needed to get the datepicker working, so I downloaded jquery-ui, swiped the css file (un-minified), and added it to my client/stylesheets folder.

3. Add jQuery-ui images

The css styled the datepicker calendar perfectly, except for the previous month and next month buttons. jQuery-ui comes with a few standard icons, so I placed the jQuery-ui images folder in /public, since static files must be kept in a public folder for meteor to acknowledge them.

4. Relocate the images

The jQuery-ui css file tries to locate the images in its parent folder, but as I mentioned, static assets = public folder. The advantage of the public folder is that all assets can be referenced with a leading slash: /images. I scanned the un-minified version of the css and changed the image url locations from images/... to /images/...

Voila! A beautiful, functional datepicker!

helpers calling helpers

Tue, 03 Jun 2014 00:00:00 GMT

Whittling away at the Discover Meteor tutorial for creating a social news sharing site, I was implementing a Comments collection and related accoutrements (templates, helpers, forms, etc.) when I uncovered a need to call one helper method from another.

I was using this handy method to return the number of comments made on a particular post:

Template.postPageItem.helpers({
  commentsCount: function() {
    return Comments.find({postId: this._id}).count();
  }
});

submitted by {{postAuthor}}, with
<a href="{{pathFor 'postPage'}}">{{commentsCount}} comments</a>

Which returned a very ordinary:

submitted by meImAnAwesomeUser, with 1 comments

1 comments?! That makes no sense.. I didn’t minor in English so I could let that travesty fly. I made a new helper method:

Template.postPageItem.helpers({
  commentsCount: function() {
    return Comments.find({postId: this._id}).count();
  },
  commentsNote: function() {
    return Comments.find({postId: this._id}).count() === 1 ? "comment" : "comments";
  }
});

UGH! What is that on line 6? Possibly the ugliest ternary statement I have written today (probably). Is that code duplication I smell? Won’t reusing commentsCount remove the redundancy?

Template.postPageItem.helpers({
  commentsCount: function() {
    return Comments.find({postId: this._id}).count();
  },
  commentsNote: function() {
    return commentsCount === 1 ? "comment" : "comments";
  }
});

Only.. that doesn’t work:

Exception from Deps recompute function: ReferenceError: commentsCount is not defined
at Object.Template.postPageItem.helpers.commentsNote (http://localhost:3000/client/views/posts/post_page_item.js?160ec9a926881c795377a47c15affc4f553c4d88:9:12)

But it is RIGHT ABOVE! I can see it! It is in the same file! I tried all permutations of commentsCount(), this.commentsCount, self.commentsCount, this.self.commentsCount(), and they all threw either similar errors or undefined. Calling the prototype method directly turns out to work just fine:

Template.postPageItem.helpers({
  commentsCount: function() {
    return Comments.find({postId: this._id}).count();
  },
  commentsNote: function() {
    var count = Template.postPageItem.commentsCount()
    return count === 1 ? "comment" : "comments";
  }
});

Except that when commentsCount is called on line 6, this is no longer the lovely Post item provided by the template. No, this becomes the context of the commentsNote function, and therefore calling this._id will return undefined. The closure issue can be worked around with a little argument passing:

Template.postPageItem.helpers({
  commentsCount: function(id) {
    var count_id = typeof id !== 'undefined' ? id : this._id;
    return Comments.find({postId: count_id}).count();
  },
  commentsNote: function() {
    var count = Template.postPageItem.commentsCount(this._id)
    return count === 1 ? "comment" : "comments";
  }
});

Wow. Passing in the id works, but now I have to check the context to determine the appropriate id. That is the most unreadable-but-reused code I have written today (probably), and I learned some valuable lessons on this frantic search for DRY:

self, whether called from a helper or the console, will always be window
this called from a Template helper refers to the item of the collection that is passed in to the template
because helper methods are closures, this in the calling method is different from this in the called method
don’t dry out your code to the point of unreadability

In the end, cleaning up the ugly ternary makes the code reuse more reasonable.

Template.postPageItem.helpers({
  commentsCount: function() {
    return Comments.find({postId: this._id}).count();
  },
  commentsNote: function() {
    var count = Comments.find({postId: this._id}).count();
    return count === 1 ? "comment" : "comments";
  }
});

Awesome.

flash your local collection

Mon, 26 May 2014 00:00:00 GMT

Rails allows you to pass messages to the user between actions (“This ActionDispatch::Flash will self-destruct”), and because Rails, all the message generation happens server side:

def show
  flash[:notice] = "You dropped your packet."
end

Meteor is mostly a client-side application framework, and you can implement a similar message system, which will never touch the server, by defining a collection that is local to the browser.

1. Create a local collection

Errors = new Meteor.Collection(null);

Passing null into the Collection declaration tells Meteor that the collection should not be extended to the server — it creates an “unmanaged (unsynchronized) local collection” (docs)

2. Throw an error whenever you need to

if(error){
  Errors.insert({message: error.reason});
}

3. Create a template for displaying the error

<template name="errors">
  <div class="errors">
    {{#each errors}}
      {{> error}}
    {{/each}}
  </div>
</template>

<template name="error">
  <div class="alert alert-error">
    <button type="button" class="close" data-dismiss="alert">&times;</button>
    {{message}}
  </div>
</template>

4. Create a helper method for serving the errors to the template

...
Template.errors.helpers({
  errors: function() {
    return Errors.find();
  }

Every template that contains the handlebars {{> errors}} will now display whatever errors have been saved to the session collection.

Awesome.

Resources

discovermeteor.com Meteor docs

meteor sessions

Sat, 10 May 2014 00:00:00 GMT

Session

stores information that relates to a single user’s instance of your application
“global reactive data store”: ** there is only one session per application instance ** the information in the session is as reactive as the database information store

Set a session variable

> Session.set('myVariable', 'myVariableValue');

Retrieving a session variable

> Session.get('myVariable');

Autorun

Yes, the session is a reactive data store, meaning it can be altered or added to at will. Why doesn’t it automagically update my UI? Because the context in which the Session is altered or added to may not be a reactive context. When the application is opened for the first time, the Session is read from and added to as the application code is loaded, but unless the Session changes are included in an Autorun block, meaning it is re-run every time there are changes made to it, the Session and all its fancy state-ness is only loaded the once.

Lessons about sessions

Always store user state in the Session or the URL so that users are minimally disrupted when a hot code reload happens.
Store any state that you want to be shareable between users within the URL itself.

mongo trouble? reset!

Sat, 03 May 2014 00:00:00 GMT

Back in March I wrote about letting my computer’s battery die in the middle of a meteor project, with the consequence being that I could not connect to the underlying mongo database:

$ meteor
[[[[[ ~/project/in/progress ]]]]]

=> Started proxy.
Unexpected mongo exit code 100. Restarting.
Unexpected mongo exit code 100. Restarting.
Unexpected mongo exit code 100. Restarting.
Can't start Mongo server.
MongoDB had an unspecified uncaught exception.
This can be caused by MongoDB being unable to write to a local database.
Check that you have permissions to write to .meteor/local. MongoDB does
not support filesystems like NFS that do not allow file locking.

After much spelunking in stackoverflow and documentation, I discovered the magic of this command:

$ sudo -u mongodb mongdo --repair --dbpath /var/lib/mongodb
$ sudo service mongodb start

Repairing and restarting the mongo service seemed to do the trick, and I continued on my merry meteor way. Well, I did it again — I let my battery die in the middle of a meteor project, and with feelings of smug satisfaction, I plumbed my blog archive for the answer. Only this time the magic antidote didn’t work:

$ sudo -u mongodb mongod --repair --dbpath /var/lib/mongodb/
[sudo] password for keighty:
Sat May  3 09:52:04.793 [initandlisten] finished checking dbs
Sat May  3 09:52:04.793 dbexit:
Sat May  3 09:52:04.793 [initandlisten] shutdown: going to close listening sockets...
Sat May  3 09:52:04.793 [initandlisten] shutdown: going to flush diaglog...
Sat May  3 09:52:04.793 [initandlisten] shutdown: going to close sockets...
Sat May  3 09:52:04.793 [initandlisten] shutdown: waiting for fs preallocator...
Sat May  3 09:52:04.793 [initandlisten] shutdown: closing all files...
Sat May  3 09:52:04.793 [initandlisten] closeAllFiles() finished
Sat May  3 09:52:04.793 [initandlisten] shutdown: removing fs lock...
Sat May  3 09:52:04.794 dbexit: really exiting now

$ sudo service mongodb start
mongodb start/running, process 2415
$ meteor
[[[[[ ~/project/in/progress ]]]]]

=> Started proxy.
Unexpected mongo exit code 100. Restarting.
Unexpected mongo exit code 100. Restarting.
Unexpected mongo exit code 100. Restarting.
Can't start Mongo server.
MongoDB had an unspecified uncaught exception.
This can be caused by MongoDB being unable to write to a local database.
Check that you have permissions to write to .meteor/local. MongoDB does
not support filesystems like NFS that do not allow file locking.

C’mon — It worked the last time! I tried it again with the same result. I tried chown-ing the local db. I tried chown-ing the database in /var/lib/mongodb. After each attempted solution the mongobd service would start, but meteor hiccuped on writing to the database. So I tried something radical. Something that wasn’t in any of the manuals, or any of the responses to similar questions on stackoverflow:

$ meteor reset
Project reset.
$ meteor
[[[[[ ~/project/in/progress ]]]]]

=> Started proxy.
=> Started MongoDB.
=> Started your app.

=> App running at: http://localhost:3000/

Turns out that if you hard shut-down your computer while your meteor project’s mongo instance is running, it remains in a locked mode. Since it was only a development database I took a chance that dropping it would cure my ills. Voila! Meteor and mongodb are friends again.

Awesome.

meteor, fetch!

Mon, 31 Mar 2014 00:00:00 GMT

From Discover Meteor: find() returns a cursor, which is a reactive data source.

Data from reactive sources will be updated automatically to track changes in the data used to generate it. Cursors are derived from specific sources:

Session variables
Database queries on Collections
Meteor.status
The ready() method on a subscription handle
Meteor.user
Meteor.userId
Meteor.loggingIn

&gt; Posts.find()
LocalCollection.Cursor {collection: LocalCollection, sorter: null, _selectorId: undefined, matcher: Minimongo.Matcher, skip: undefined…}
      _selectorId: undefined
      _transform: null
      collection: LocalCollection
      cursor_pos: 0
      db_objects: null
      fields: undefined
      limit: undefined
      matcher: Minimongo.Matcher
      reactive: true
      skip: undefined
      sorter: null
      __proto__: Object

Cursors are not query snapshots. They are query snapshot containers — the contents of which are updated by changes to the underlying data. To interact with the data contained in a cursor, we can use fetch() to transform the cursor into an array.

&gt; Posts.find().fetch()
[
    Object
    _id: "z4z36DFCaGr8Ptuqd"
    title: "The Way We Live Now"
    author: "Anthony Trollope"
    __proto__: Object
    ...
]

Meteor can iterate over a cursor without first converting it to an array, but in order to interact with the data directly, you must first use fetch(), map(), or forEach(). For more cursor fun, checkout the Meteor docs.

Awesome.

situations matter

Fri, 28 Mar 2014 00:00:00 GMT

Situations Matter by Sam Sommers is an eye-opening study of the influence of context on human interactions. Social psychologists call it “fundamental attribution error”, which is a tendency to interpret someone else’s behavior as stemming from their core characteristics, rather than to external circumstances. A simplistic example of attribution error is assuming that the person driving the speeding car is a reckless speed-freak rather than someone rushing a woman in labor to the hospital. Sommers has studied the power of situations over human interactions and how the attribution error interferes with our ability to empathize with the unfamiliar. I have read similar studies of how bias infiltrates our relationships, but what surprised me the most is how it also influences our perception of the self.

You’re not the person you thought you were

A provocative but aptly named chapter, Sommers overturns the concept of an “authentic self” with a simple fill-in-the-blanks exercise:

I am _
I am _
I am _
I am _
I am _

His study subjects were required to fill in 20 of these blanks, but 5 is enough to illustrate his point to the reader. The open-endedness of this exercise allows for a fantastic range of responses: “I am 36 years old, I am hungry, I am tall, I am a lifelong learner”. Taking the 20 questions test in different contexts will yield different definitions of yourself. Sommers has discovered that “we tend to think of ourselves in terms of that which makes us distinctive”, so your answers will have more to do with the people around you than with a fixed set of characteristics. A woman is more likely to include gender on her list if she is in a room full of men than if she is in a room full of women. So it seems that “identity is malleable and personal preferences are constructed on the spot.” Introspection will gain us little but a hazy snap-shot of our identity at any given moment.

“It’s refreshing to realize that you’re not a finished product — that who you are in the here and now may not be the same person you’ll be in the then and there. In fact, it’s the opposite view of the self as a fixed entity that causes problems. When you assume that there’s a true core self waiting to be discovered, that’s when your potential seems limited and the world around you is full of threats to be rationalized away.”

One quality that is typically presumed to be fixed is intelligence. Challenging this truism, another study was conducted where students were grouped according to their belief that either intelligence is a stable attribute, or that their intellect can be changed. The researchers asked the students to predict the likelihood that they will attend a remedial language course, and then tracked their performance. It was found that among the students who fared poorly in the language course, those who viewed intelligence as a permanent, unchangeable trait were less likely to enroll in the remedial course — that is, to take steps in order to improve their performance. For students who believe that intelligence is fixed, a poor exam grade is a mark of their limitations rather than an opportunity for improvement. Students who believe that intelligence is malleable are more likely to show the stick-to-itiveness necessary for self-improvement.

Viewing the self less like a rigid Rubik’s Cube to be solved, and more like Silly Putty that conforms to the shape of its container, will likely free up a lot of my time. Instead of trying to solve the puzzle. I can spend energy on finding the container that makes me happy… for the moment.

the ambition gap - leaning in

Sun, 23 Mar 2014 00:00:00 GMT

I may be a little late to the game, but I am finally reading Sheryl Sandberg’s “Lean In”, and right from the introduction I can understand why it caused such a stir when it came out in 2012. Sandberg addresses the issue of why there are relatively few women in leadership roles, and ways that women have held themselves back. Women earn more than 57% of undergraduate degrees and 60% of masters degrees, and yet hold less than 14% of C level positions. I always assumed that as women disproportionately fill the ranks of entry level positions, there would be a proportional increase of women filling senior level positions as well. Apparently this is not the case. “The exodus of highly educated women is a major contributor to the leadership gap.”

Sandberg shares a story about a kindergarten class where the students were asked “What do you want to be when you grow up?“. Several of the boys wanted to be president, but none of the girls did. This disparity continues into middle school, where more boys than girls aspire to leadership roles in future careers. {“36% of men want to reach the C-suite and only 18% of women.“} Why so few? If the question were ever posed to me I would honestly say that rising to the C-levels has never been of any interest, and I think Sandberg is right to ask why.

Too much Barbie or not enough Hillary?

Sandberg is quick to link the relative derth of women in leadership roles on the messages of obedience and passivity we instill in girls and the general indulgence of opposite traits in boys (“boys will be boys”). “Gender stereotypes introduced in childhood are reinforced throughout our lives and become self-fulfilling prophesies.” While gender stereotypes have held women back in the past, today the difference is in their perceived options, not in perceived abilities. When the kindergarten-ers in Sandberg’s story were asked what they wanted to be when they grew up they looked to the immediate role models around them. Call it a lack of imagination about one’s future career, rather than a lack of ambition.

She has recently begun a campaign to ban the word “bossy”, and while words have the power to shape our values, it is a passive approach to solving the problem. Boys don’t show a propensity towards leadership in middle-school any more than girls show a propensity towards secretarial work. Sandberg’s “Lean In” project comes closer to directly addressing the issue. She realizes that you can’t be what you can’t see”, and works to raise the profiles of powerful women. When it becomes normal to have female leaders, I am sure we will find more girls expressing ambition. Likewise, when it becomes normal for men to share equally in care-giving, boys will begin making different choices.

Lean In: Women, Work, and the Will to Lead by Sheryl Sandberg

line, spline

Thu, 20 Mar 2014 00:00:00 GMT

D3 lines are rendered using linear interpolation by default. This means that the line connecting a series of points will trace a direct path from one node to another:

The curved line is rendered with the cardinal interpolator, one of several options D3 provides. The curvature of the line is called a spline. Splines are tools used by architects to draw curved lines, and the term was adopted by mathematicians to describe smooth, piecewise polynomial approximation:

Splines are curves, which are usually required to be continuous and smooth… [The] join points are called knots. Splines with few knots are generally smoother than splines with many knots; however, increasing the number of knots usually increases the fit of the spline function to the data. Knots give the curve freedom to bend to more closely follow the data. —Wikipedia

multiple remotes

Wed, 19 Mar 2014 00:00:00 GMT

Somehow I had worked out in my head that there could be only one remote. Origin. Turns out you can set and push to a (presumably) infinite number of remote repositories :

$ git config --list
push.default=nothing
alias.hist=log --pretty=format:"%h %ad | %s%d [%an]" --graph --date=short
user.name=Katie Leonard
…
remote.origin.url=git@github.com:keighty/thorfiles.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master

(On a side note, I am not sure I remember putting alias.hist in my git config, but I find it to be wickedly useful.)

Setting a new remote is as easy as:

$ git remote add new_destination git@github.com:otheruser/thorfiles.git

Standing back to admire the results:

$ git config --list
push.default=nothing
alias.hist=log --pretty=format:"%h %ad | %s%d [%an]" --graph --date=short
user.name=Katie Leonard
…
remote.origin.url=git@github.com:keighty/thorfiles.git
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
branch.master.remote=origin
branch.master.merge=refs/heads/master
remote.new_destination.url=git@github.com:otheruser/thorfiles.git
remote.new_destination.fetch=+refs/heads/*:refs/remotes/new_destination/*

Fast-forward through some excellent and brilliant feature work on my_branch, and I naturally want to save it for posterity:

$ git push origin my_branch
Counting objects: 611, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (263/263), done.
Writing objects: 100% (375/375), 63.40 KiB | 0 bytes/s, done.
Total 375 (delta 280), reused 189 (delta 105)
To git@github.com:keighty/thorfiles.git
 * [new branch]      my_branch -> my_branch

I can also share it with my friend’s repo, presuming I have write access, and knowing full well he will want to use my new feature:

$ git push new_destination my_branch
Counting objects: 611, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (263/263), done.
Writing objects: 100% (375/375), 63.40 KiB | 0 bytes/s, done.
Total 375 (delta 280), reused 189 (delta 105)
To git@github.com:otheruser/thorfiles.git
 * [new branch]      my_branch -> my_branch

Git is so awesome.

delay chained transitions

Sat, 15 Mar 2014 00:00:00 GMT

Changing an element using multiple, chained transitions requires a little troubleshooting. You would think that just chaining them one after another would suffice:

selection.transition().duration(1500)
    .attr("cx",getX)
    .attr("cy",getY)
    .attr("r", getR)
    .style("fill", getColor)
  .transition().duration(1500)
    .attr("r", doubleR);

But D3 does not wait for one transition to be complete before it begins the next one. Adding a delay will accomplish the desired transition chaining:

selection.transition().duration(1500)
    .attr("cx",getX)
    .attr("cy",getY)
    .attr("r", getR)
    .style("fill", getColor)
  .transition().delay(1700).duration(1500)
    .attr("r", doubleR);

easing does it

Wed, 12 Mar 2014 00:00:00 GMT

With the possible exception of UFOs, things don’t usually switch from being stationary to moving at maximum velocity without an intervening period of acceleration or deceleration. Similarly, it can be difficult to track elements on a web page that move from one position to another with no warning, or faster than the human eye can perceive. Enter easing functions:

”Easing functions specify the rate of change of a parameter over time.” —Robert Penner

selection.enter()
  .append("circle")
  .attr("cx",scaleX(0))
  .attr("cy",scaleY(0))
  .attr("r", 0);
selection.transition().ease("cubic-in-out").duration(1500)
  .attr("cx",getX)
  .attr("cy",getY)
  .attr("r", getR)
  .style("fill", getColor);

The transition will act on the selection in the following ways:

it will move the element from its starting x position of 0 to the x position defined by getX()
it will move the element from its starting y position of 0 to the y position defined by getY()
it will increase the element’s radius from 0 to the radius returned from getR()
it will gradually change the element’s color using an ordinal color scale, from black to the color returned from getColor()
all of this movement will be performed with a “cubic-in-out” easing function

The cubic-in-out easing function is the standard D3 easing function. The element will increase velocity by a power of 3 until halfway down the path it is destined to trace. It will then decrease velocity by a power of 3 until it reaches its final location. There are many different types of easing functions. Check out the resources for more information, and ask your doctor which easing function is right for you!

Resources

Robert Penner’s Easing Functions

Data Visualization with D3.js Cookbook (affiliate link)

git, what have I done?

Sat, 08 Mar 2014 00:00:00 GMT

Git, the superhero of version control, allows you to review all the commands that have modified the HEAD tag in a git repo: git reflog

The output for one of my demo apps:

5075d26 HEAD@{0}: checkout: moving from master to kleonard/add_skyline_background
531c396 HEAD@{1}: pull: fast forward
435f6de HEAD@{2}: commit: Adds logic for running and testing
043e35a HEAD@{3}: commit: Adds logic to process input
5155ba4 HEAD@{4}: commit: Adds input file
a629871 HEAD@{5}: rebase finished: returning to refs/heads/kleonard/create_input_files
a010021 HEAD@{6}: rebase: Add input file
b7cd86a HEAD@{7}: commit: Initial commit arbitrage
5ed00ea HEAD@{8}: commit: Adds launcher
7b2d720 HEAD@{9}: commit: Adds logic to find smallest number of moves
9c0dfce HEAD@{10}: commit: Adds implementation for bin creation

Hmmm… looks like my commit messages need a little work… Still, it is a great tool for catching up on the changes to a repo (especially if you haven’t touched it in a while), finding out where the bulk of the work is happening, or just reminding yourself what you did yesterday.

Awesome.

the value of slow transitions

Sat, 08 Mar 2014 00:00:00 GMT

The Why

From Object Constancy by Mike Bostock:

Animated transitions are pretty, but they also serve a purpose: they make it easier to follow the data. This is known as object constancy: a graphical element that represents a particular data point… can be tracked visually through the transition. This lessens the cognitive burden by using preattentive processing of motion rather than sequential scanning of labels.

(The emphasis is mine)

The How

Object constancy is achieved through object identity: the data is bound to it’s object by a shared unique identifier. This allows D3 to reuse the object when the data set is updated. Object identity is achieved by passing a key function to the data() method at the time of data binding:

var data = [ // <-A
        {id: 0, value: "foo"},
        {id: 1, value: "bar"},
        {id: 2, value: "foo"},
        {id: 3, value: "baz"},
        {id: 4, value: "bar"}
    ];
d3.selectAll("svg:g")
    .data(data, function(d){return d.id;});

transition with style

Thu, 06 Mar 2014 00:00:00 GMT

Making things zoom across the page is what I used to think javascript was all about, but it wasn’t until I started learning d3 that I found out how to make it happen.

var body = d3.select("body"),
    duration = 2000;

body.append("div")
        .style("background-color", "white")
        .style("margin", "40px")
        .style("width", "200px")
        .style("height", "200px")
        .style("border", "black solid thin")
    .transition()
    .duration(duration)
        .style("background-color", "black")
        .style("margin-left", "600px")
        .style("width", "100px")
        .style("height", "100px");

D3 is smart enough to select an appropriate scale for the property in transition. If a transitioning element is not given an explicit starting point, D3 will use the computed style. If the end value is missing, the property will be treated as a constant. In the example above, the border stays the same, as do the top and bottom margins. The only properties that change are the ones specified below the transition().

Awesome.

Resources

Data Visualization with D3.js Cookbook (affiliate link)

fun with ticks

Wed, 05 Mar 2014 00:00:00 GMT

D3 has an awesome tick library, which makes it easy to customize an axis. I have not found any evidence of a built in function to customize tick rendering depending on if it is a major or a minor tick. One way to get around that is to render two axes, and only label one:

var scaleX=d3.scale.linear().domain([0,10]).range([margin,width-margin]);

var svg=d3.select("body").append("svg")
      .attr("width",width)
      .attr("height",height);

var xAxis = d3.svg.axis()
  .scale(scaleX)
  .orient("bottom")
  .ticks(5)
  .tickSubdivide(5)
  .tickPadding(10)
  .tickFormat(function(d) { return d + " ticks"});

var x2Axis = d3.svg.axis()
  .scale(scaleX)
  .orient("bottom")
  .ticks(5)
  .tickSize(14)
  .tickFormat("");

svg.append("g")
  .attr("class", "axis")
  .attr("transform", "translate(0," + (margin) + ")")
  .call(xAxis);

svg.append("g")
  .attr("class", "axis")
  .attr("transform", "translate(0," + (margin) + ")")
  .call(x2Axis);

TADA

More fun with ticks

orient(): set which side of the axis will be ticked and labeled
ticks(): set the number of major ticks
tickSubdivide(): set the number of minor ticks
tickPadding(): set the distance between the ticks and their label
tickFormat(): set the units to appear after the label on the major ticks
tickSize(): specify one arg for inner ticks, another argument for outer ticks, or use outerTickSize() or innerTickSize() to specify one or the other. Outer ticks are at the ends of the scales.

a technique for producing ideas

Tue, 04 Mar 2014 00:00:00 GMT

A Technique for Producing Ideas by James Webb Young (1965)

Young proposes that all ideas are the result of five identifiable stages of rumination, whether they are followed consciously or not:

Gather information, and not just what is specific to your problem domain. Novelty comes from mashing seemingly unrelated concepts together.
Review the information — collate your information in some fashion and go over it. A lot.
Do something else. Get distracted from whatever problem you are trying to solve.
“Eureka! I have it!”
Refine your idea — first drafts are never perfect, so keep polishing.

ordinal scale

Tue, 04 Mar 2014 00:00:00 GMT

d3 nests and rollups

Tue, 04 Mar 2014 00:00:00 GMT

D3 provides powerful functions for manipulating raw data sets, including csv(), nest(), and rollup().

I generated a hundred rows of test_data.csv using generatedata.com:

Colors,X,Y,Quantity
violet,1,8,91
blue,9,1,32
blue,3,1,67
yellow,5,5,63
violet,6,6,57
green,5,10,65
red,8,2,36
...

The basic properties are a Color (one of blue, yellow, violet, green, and red), an X-value (between 1 and 10), a Y-value (also between 1 and 10), and a Quantity value (between 1 and 100).

csv()

You can import a raw, comma-delimited data file, and bind the data directly to an svg.

d3.csv("test_data.csv", function(csv) {

  svg.selectAll("circle").data(csv).enter()
    .append("circle")
    .attr("cx",getX)
    .attr("cy",getY)
    .attr("r", getR)
    .style("fill", getColor);
})

nest() and rollup()

You can also aggregate data on any property — just tell D3 which one, and use any of D3’s handy utility functions (mean, median, sum, min/max, quantile, etc) to return the aggregate data you want to display. For example, I want to plot the average data for each color. I tell my D3 selection to nest the data on the color key. Then I tell the rollup function to take the average for each property of the data. Pass the new data to the svg instead of the csv, and voila

d3.csv("test_data.csv",function(csv) {
  data = d3.nest()
          .key(function(d) { return d.Colors; })
          .rollup( function(d) {
            return {
              "X": d3.mean(d, function(e) { return +e.X; }),
              "Y": d3.mean(d, function(e) { return +e.Y; }),
              "Quantity": d3.mean(d, function(e) { return +e.Quantity; })
            };
          })
          .entries(csv);

  svg.selectAll("circle").data(data).enter()
    .append("circle")
    .attr("cx",getX)
    .attr("cy",getY)
    .attr("r", getR)
    .style("fill", getColor);

})

D3 is so awesome.

scaling d3

Mon, 03 Mar 2014 00:00:00 GMT

Add scales to a d3 plot in three easy steps:

1. Define the scale

// Image dimensions
var width = 500,
    height = 500,
    margin = 50;

var scaleX=d3.scale.linear().domain([0,10]).range([margin, width-margin]);
var scaleY=d3.scale.linear().domain([0,10]).range([height-margin, margin]);

2. Draw the scale

// Image container
var svg=d3.select("body").append("svg")
      .attr("width",width)
      .attr("height",height);

// Axis definition
var xAxis = d3.svg.axis()
  .scale(scaleX)
  .orient("bottom");

var yAxis = d3.svg.axis()
  .scale(scaleY)
  .orient("left");

// Axis graphics
svg.append("g")
  .attr("class", "axis")
  .attr("transform", "translate(0," + (height - margin) + ")")
  .call(xAxis);

svg.append("g")
  .attr("class", "axis")
  .attr("transform", "translate(" + margin + ",0)")
  .call(yAxis);

3. Apply the scale to your data

d3.csv("test_data.csv",function(csv) {

  svg.selectAll("circle").data(csv).enter()
    .append("circle")
    .attr("cx",getX)
    .attr("cy",getY)
    .attr("r", getR)
    .style("fill", getColor);

  // Apply scale to the x value
  function getX(d) {
    return scaleX(d.X);
  }

  //Apply scale to the y value
  function getY(d) {
    return scaleY(d.Y);
  }

  function getR(d) {
    return d.Quantity / 10;
  }

  function getColor(d) {
    return d.Colors;
  }
}

Awesome.

javascript dates

Sat, 01 Mar 2014 00:00:00 GMT

Javascript Date objects have zero indexed months but not dates.

> (new Date(2014, 0, 1)).strftime("%b %d, %Y")
"Jan 1, 2014"
> (new Date(2013, 11, 31)).strftime("%b %d, %Y")
"Dec 31, 2013"
> (new Date(2014, 0, 0)).strftime("%b %d, %Y")
"Dec 31, 2013"

Wat?

Resources

Javascript strftime

binding functions as data

Thu, 27 Feb 2014 00:00:00 GMT

Primitives and object literals are not the only things you can bind to a D3 visualization. After all, aren’t javascript functions objects as well?

var data = [];
var next = function(x) { return 15 + x*x };
var newData = function() { data.push(next); return data; }
var selection = d3.select("#container")
                  .selectAll("div")
                  .data(newData);

...
                  .select("#span")
                    .text(function(d, i) { return d(i); });

If you provide a function to the data function, D3 will simply invoke the function and use the returned value as a parameter of the data function.

Resources

Data Visualization with D3.js Cookbook (affiliate link)

d3 array functions

Thu, 27 Feb 2014 00:00:00 GMT

D3 provides utility functions to perform common operations on Javascript arrays:

min
max
extent (range)
sum
median
mean
sort
quantile
bisect

The Neuroscience of Programming

Thu, 27 Feb 2014 00:00:00 GMT

Scientists Begin Looking at Programmers’ Brains: The Neuroscience of Programming from The Huffington Post.

A new study provides new evidence that programmers are using language regions of the brain when understanding code and found little activation in other regions of the brain devoted to mathematical thinking.

Each programming language you learn requires expressing abstract concepts with a different syntax — and in the case of punctuation (~>, =>, :, {}, [], etc) an augmented alphabet. How is a programming language NOT like linguistics?

Besides a mathematical inclination, an exceptionally good mastery of one’s native tongue is the most vital asset of a competent programmer. —Djikstra

string conversion in javascript

Thu, 27 Feb 2014 00:00:00 GMT

Almost every day, javascript blows my mind. Today I discovered that you can convert a string to a number simply by prepending a +:

> typeof(+"this is a string")
"number"
> typeof(+"100")
"number"
> +"100" + 50
150

Awesome.

implied functions applied

Wed, 26 Feb 2014 00:00:00 GMT

When binding a primitive datum to a node in a D3 visualization, the callback function can be implicitly called on the datum without the typical verbosity:

svg.selectAll("circle")
    .data([32, 57, 112, 293])
  .enter().append("circle")
    .attr("cy", 90)
    .attr("cx", String)
    .attr("r", Math.sqrt);

In line 5, String() is performed on the data element, as is Math.sqrt in line 6, with the same result as:

svg.selectAll("circle")
    .data([32, 57, 112, 293])
  .enter().append("circle")
    .attr("cy", 90)
    .attr("cx", function(d) {
      return String(d);
    })
    .attr("r", function(d) {
      return Math.sqrt(d);
    });

… but better.

Awesome.

Resources

Three Little Circles D3 Tutorial

visualize with purpose

Wed, 26 Feb 2014 00:00:00 GMT

Information visualization (insight, understanding, and discovery) and information art (aesthetics) differ in their context, audience, and goals. In my exploration of D3 I came across an Information Visualization Manifesto by Manuel Lima, and this is what I have learned:

Form follows function: keep your problem domain in mind when assembling a visualization.
Question: Drive your work through questions and curiosity.
Interactivity is key.
Sources: Provide access to the raw data if possible.
Narrative: Tell a story with your data.
Aesthetics Are a consequence of visualization, not a goal.
Relevancy: What is the context? What patterns are discernable?
Time: Consider time when your data is affected by it.
Knowledge: Bring new understanding and insight.
Ease up: Not everything can or should be visualized. Be deliberate and add to information, don’t obfuscate with unnecessary visualizations.

enter update exit

Tue, 25 Feb 2014 00:00:00 GMT

D3 is a data visualization tool that uses method chaining to produce big pictures with little code. The selection library performs a full Outer Join between the data (D) and the visual elements (V), and the enter-update-exit pattern performs work on subsets of this join:

UPDATE: Inner Join (D union V)

selection.data(data_points)

data() binds data_points to the html or svg elements returned by the selection and returns the data not visualized.

ENTER: Left excluding join (D/V)

selection.data(data_points).enter()

enter() returns data_points that are not yet represented visually by html or svg. Chain .append() to add a new element to the page.

EXIT: Right excluding join (V/D)

selection.data(data_points).exit()

exit() returns html or svg elements that no longer have data associated with them. Chain .remove() to eliminate them, or .translate() to make them slide off the page.

Resources

Data Visualization with D3.js Cookbook (affiliate link)

Visual Representation of SQL Joins

d3 is selective

Fri, 21 Feb 2014 00:00:00 GMT

D3 uses the W3C standard selector library, and selection modifiers work on both single and multiple element selections, and will automatically apply the modifier iteratively over the entire collection, which makes your code simpler and easier to maintain.

Selection modifiers

d3.select(‘p’).text() : retrieve or modify text
d3.select(‘p’).attr() : retrieve or modify a given attribute
d3.select(‘p’).classed() : retrieve or modify css classes
d3.select(‘p’).style() : retrieve or modify styles
d3.select(‘p’).html() : retrieve or modify a selections inner html (like text but with tags)

d3.selectAll('div')
            .attr("class", "alert")
            .each(function(d, i) {
              d3.select(this).append("h1").text(i);
            });

Selections are essentially arrays with some extra variables baked in. The .each call on line 3 treats the selected elements as an array, and performs a function on each one. The extra variables provided to the iterator function for free are: the data bound to the element (d), the index of the element, and the element itself (this).

Awesome

functional arguing

Thu, 20 Feb 2014 00:00:00 GMT

The arguments variable is a hidden parameter in javascript functions that contains all the arguments for the invocation in a pseudo-array (ie: it does not have array functions other than length and indexing).

instance.headline = function (h) {
  if (!arguments.length) return headline;
  headline = h;
  return instance;
};

Even though the function specifically calls for an argument (h), it can still be called without one. This construction has a ruby equivalent, attr_accessor :headline, which behaves as a getter and a setter, depending on the arguments provided (none, or headline = new_value).

This “variable-parameter” function design is what enables method chaining. When the function is called as a getter it returns a string, but when it is called as a setter it returns the invoking object, passing itself along to the next method:

var widget = SimpleWidget({ color: "#6439ed" })
                .headline("SimpleWidget")
                .description("This is a simple widget")
                .render();

Resources

Data Visualization with D3.js Cookbook (affiliate link)

how to live on 24 hours a day

Mon, 17 Feb 2014 00:00:00 GMT

How to Live on 24 Hours a Day, by Arnold Bennett (published in 1910)

Arnold Bennett’s recommendations for leading a full and happy life:

Wake up earlier
Practice concentrating your mind on a single task.
Don’t try to change too much too quickly.
Spend time each day reflecting on how closely your life aligns with your principles.
Study the performing arts.
Read poetry.
Failing that, read anything except novels, and think about what you read.
Don’t be smug about it.

I was surprised to learn that much of the ennui of modern life turns out not to be particularly modern after all. In this often sarcastic essay, Bennett shares his recipe for maximizing “the full use of one’s time to the great end of living (as distinguished from vegetating).”

Wake up earlier

In the preface he disposes of the most common excuse for not improving oneself — I just don’t have enough time. The solution is simple:

Employ your engines in something beyond [your routine] before, and not after, you employ them on the [routine] itself. Briefly, get up earlier in the morning… Most people sleep themselves stupid.

This call to action comes before he even outlines his ideas for how you should spend your reclaimed time. Bennett asks the reader whether they have not ever said to themselves “I would love to do such-and-such if only I had the time,” or “When such-and-such is over I will have time to address this-and-that?” We all have the wish to accomplish something beyond our routine, and it will give us no peace (but an awful lot of guilt) until we have done some small thing towards the pursuit of that goal.

Time is unstealable, and is measured out to you a moment at a time without regard to whether you use it wisely, squander it, or deserve it. No amount of birthright, money or genius will increase your allotment. Why spend it sleeping?

Practice concentrating your mind on a single task.

You practice physical exercises for a mere ten minutes morning and evening, and yet you are not astonished when your physical health and strength are beneficially affected every hour of the day, and your whole physical outlook changed. Why should you be astonished that an average of over an hour a day given to the mind should permanently and completely enliven the whole activity of the mind?”

My mind wanders constantly, even though I have read so much about willpower and meditation. Disciplining the mind has to be the difficulty Bennett warned us of, but he claims that “mind control is the first element of a full existence:

Nothing whatever happens to us outside our own brain; since nothing hurts us or gives us pleasure except within the brain, the supreme importance of being able to control what goes on in that mysterious brain is patent.

Don’t try to take on too much too quickly

Beware of undertaking too much at the start. Be content with quite a little. Allow for accidents. Allow for human nature, especially your own.

At New Year’s time we make resolutions to eat healthier and exercise more, to spend more time with our families and to watch less TV. These are all laudable goals, but they mostly go out the window before the last day of January. Bennett’s advice - “let the pace of the first lap be even absurdly slow, but let it be as regular as possible.” Slow and steady wins the race.

Spend time each day reflecting on how closely your life aligns with your principles.

All martyrs are happy, because their conduct and their principles agree. [Perform a] daily, candid, honest examination of what one has recently done, and what one is about to do — of a steady looking at one’s self in the face (disconcerting though the sight may be).

Study the performing arts.

Reading literature is not the only pursuit that will expand your appreciation of the finer things in life. If you enjoy music, pick up a book on how to appreciate music. Then, the next time you found yourself at a concert “you would live [there], whereas previously you had merely existed there in a state of beatific coma, like a baby gazing at a bright object.

Read poetry

A good novel rushes you forward like a skiff down a stream, and you arrive at the end, perhaps breathless, but unexhausted. The best novels involve the least strain. Now in the cultivation of the mind one of the most important factors is precisely the feeling of strain, of difficulty, of a task which one part of you is anxious to achieve and another part of you is anxious to shirk; and that feeling cannot be got in facing a novel. Imaginative poetry produces greater mental strain than a novel.

Bennett has two suggestions beginning a study of any kind: the first is to define the direction and scope of your efforts, the second suggestion is to think as well as to read.

Unless you give at least forty-five minutes to careful, fatiguing reflection (it is an awful bore at first) upon what you are reading, your ninety minutes of a night are chiefly wasted. This means that your pace will be slow.

Don’t be smug about it

It is easy to begin judging people who don’t follow the same program of rigorous mind expansion as you have:

Unconsciously to become a prig is an easy and a fatal thing.
It is easy to go overboard:

A programme of daily employ is not a religion.
Don’t get carried away by the list of things to learn, of being gradually more and more obsessed by what one has to do next.
Don’t try to do too much too quickly

Let the pace of the first lap be even absurdly slow, but let it be as regular as possible.
Study what you are interested in

In choosing [what to study,] be guided by nothing whatever but your taste and natural inclination.

What an awesome find.

custom asides

Sat, 15 Feb 2014 00:00:00 GMT

I was following a tutorial on how to include a twitter widget in the sidebar of my octopress site and couldn’t figure out why my twitter content wouldn’t render. It turns out that custom asides require an internal div. The process:

1. Create a twitter timeline widget

This was surprisingly easy. Just login to twitter.com and click the settings icon:

Click “Widgets” on the bottom of the left nav and follow the instructions.

CAKE WALK.

2. Create a custom aside

Perhaps it is the tutorial showing its age, but I couldn’t get the content to render as described. I grabbed the widget html and created a file in source/custom/asides:

<section>
  <h1>Twitter</h1>
  <!-- paste twitter widget code here -->
</section>

I looked to the other asides for help, and realized that the content had to be anchored to a div of its own:

<section>
  <div>
    <h1>Twitter</h1>
    <!-- paste twitter widget code here -->
  </div>
</section>

3. Add the aside to your _config.yml

What my _config.yml looks like

# list each of the sidebar modules you want to include, in the order you want them to appear.
# To add custom asides, create files in /source/_includes/custom/asides/ and add them to the list like 'custom/asides/custom_aside_name.html'
default_asides: [custom/asides/contactme.html, asides/recent_posts.html, asides/github.html, custom/asides/aboutme.html, custom/asides/twitter.html]

Awesome.

for score

Tue, 11 Feb 2014 00:00:00 GMT

The for in statement can function as a quick and semantic iterator for looping over object properties.

In your console, try

> var person = {
    first_name: "Katie",
    last_name : "Leonard"
  };

> var name;

> for(name in person){
    console.log(name + " : " + person[name];
  )};

first_name : Katie
last_name : Leonard

Awesome

Resources

JavaScript: The Good Parts

the passionate programmer

Mon, 10 Feb 2014 00:00:00 GMT

The Passionate Programmer: Creating a Remarkable Career in Software Development (Pragmatic Life), by Chad Fowler (affiliate link)

This book is aptly titled, and an awesome read. It consists of a series of tips — strategies Fowler has employed to build his own remarkable career. The most important tip is one that can’t really be crossed off a list, and that is to build a successful career in software development one must be passionate about software development.

Update:

Unbeknownst to me when I was finishing this book, but knownst to me now is that there has been a vibrant conversation occurring on the interweb about passion and programming. Check out Avdi Grimm’s latest, The Passion Gospel.

augmented objects

Sun, 09 Feb 2014 00:00:00 GMT

Because it bears repeating: javascript and ruby have many features in common, including the ability to add functionality to existing objects.

Ruby singleton

You can add methods to an instance of a class without affecting any other instances of that class.

> x, y = "stringA", "stringB"
> def x.baz
>   puts "I can haz?"
> end
# => nil
> x.baz
I can haz?
# => nil
> y.baz
NoMethodError: undefined method `baz' for "stringB":String
  from (irb):62
  from /home/katherine/.rvm/rubies/ruby-2.0.0-rc1/bin/irb:12:in `<main>'

Javascript augmenting

You can add methods to an of an Array object without affecting any other array objects.

> var data1 = [1, 2, 3, 4, 5, 6];
undefined
> var data2 = [2, 4, 6, 8, 10, 12];
undefined
> data1.total = function() { return 21; };
function () { return 21; }
> data1.total();
21
> data2.total()
TypeError: Object [object Array] has no method 'total'

augmented types

Sat, 08 Feb 2014 00:00:00 GMT

Javascript and Ruby have many features in common, including the ability to add functionality to existing objects.

Ruby monkeypatching

Ruby allows you to reopen classes to add new functionality. For example, I can reopen the String class to add a new method, “foo()“:

irb > class String
irb >   def foo
irb >     "I see you!"
irb >   end
irb > end
# => nil
irb > "".class
# => String
irb > "".foo
# => "I see you!"

Javascript augmenting

Javascript allows you to augment existing objects with additional functionality.

> String.prototype.foo = function foo() {
    return "I see you!";
  }
function foo() {
    return "I see you!";
  }
> typeof("");
"string"
> "".foo()
"I see you!"

global abatement

Fri, 07 Feb 2014 00:00:00 GMT

or, What the heck is global abatement?

tldr: Global variables are code smells. Declaring variables inside an application-level function serves to namespace the variables and minimizes the use of global variables.

Javascript has function-level scoping, which means that variables declared anywhere within a function are scoped to that function. Writing a javascript without an enclosing function leaves your declared variables vulnerable to name collisions, or reassignment by other javascripts.

A global abatement is a strategy used to reduce the vulnerability of your javascript variables by removing them from the global scope. Application namespacing is one such strategy:

person = {
  first_name: 'Hello',
  last_name: 'Kitty'
};

var MYAPP = {};
MYAPP.person = {
  first_name: 'Hello',
  last_name: 'Kitty'
};

Declaring an application-level function ensures that the correct person is called when needed. It improves readability by essentially flagging the scope of the variable and reducing ambiguity.

JavaScript: The Good Parts

fanout

Thu, 06 Feb 2014 22:38:12 GMT

Continuing my exploration of message queues…

A common MQ modality is publish/subscribe. A simplistic queue model has producers publishing messages directly to a queue and consumers taking messages off a queue. In the publish/subscribe model, the producer sends messages to an exchange, who’s only job is to publish messages to queues. In this scenario, the exchange knows to which queue(s) the message should be published according to established rules. The relationship between an exchange and a queue is called a binding.

Types of exchanges

direct
topic
headers
fanout - broadcasts every message it receives to all the queues it knows

Create a fanout exchange:

# create and start the connection
connection = Bunny.new()
connection.start
# create and start the channel
channel = connection.create_channel
# create the exchange:
exchange = channel.fanout('logs')
# publish to your named exchange:
exchange.publish(msg)
# close the connection
connection.close

Create a fanout worker

# create and start the connection
connection = Bunny.new()
conn.start
# create and start the channel
channel  = connection.create_channel
# name the exchange to listen to
exchange = ch.fanout("logs")
# name the queue
q = ch.queue("", :exclusive => true)
# bind the queue to the exchange
q.bind(exchange)
# subscribe to the bound queue
q.subscribe(:block => true) do |delivery_info, properties, body|
  puts " [x] #{body}"
end

In action

Start three terminals:

$ ruby -rubygems receive_logs.rb > logs_from_rabbit.log
$ ruby -rubygems receive_logs.rb
$ ruby -rubygems emit_log.rb

In the last terminal, add a few more messages:

$ ruby -rubygems emit_log.rb this is a longer message
$ ruby -rubygems emit_log.rb this message should be logged in a file AND appear in my terminal window

The output in terminal 2:

$ ruby -rubygems receive_logs.rb
 [*] Waiting for logs. To exit press CTRL+C
 [x] Hello World!
 [x] this is a longer message
 [x] this message should be logged in a file AND appear in my terminal

The output from the log file:

$ cat logs_from_rabbit.log
 [*] Waiting for logs. To exit press CTRL+C
 [x] Hello World!
 [x] this is a longer message
 [x] this message should be logged in a file AND appear in my terminal

Resources

RabbitMQ.com

first in, first out

Wed, 05 Feb 2014 19:14:30 GMT

Why use message queues?

Message queues offer the benefits of high reliability at the cost of performance. This is particularly useful for web tasks that may take longer than the typical HTTP request. Requests are persisted, acknowledged, and confirmed asynchronously. They are not dropped or timed out, no matter how long they take to process.

Asynchronous request processing with RabbitMQ

A program that sends messages is a producer. A producer drops a message onto a queue (like a mailbox). A consumer is a program that mostly waits to receive messages, and processes them when they arrive.

RabbitMQ provides message acknowledgments. An ack(nowledgement) is sent back from the consumer to tell the queue that a particular message has been received and processed, and that the queue is free to delete it

Lets be fair

By default, RabbitMQ will dispatch messages equitably to all workers, and will not check first to see if the worker is already busy. You can ask the queue to only send messages to a worker if it is available to process a message.

Use channel.prefetch(1) to prevent dispatching a new message to a worker until it has processed and acknowledged the previous one. Instead, RabbitMQ will dispatch it to the next worker that is not still busy. One possible ramification of prefetching is that the queue can become backed up if all workers are busy with long processes, so watch your queue length!

Useful Commands

list active queues: sudo rabbitmqctl list_queues
list queues along with unacknowledged messages: sudo rabbitmqctl list_queues name messages_ready messages_unacknowledged

Resources

Afraid of commitment

Tue, 04 Feb 2014 16:04:27 GMT

or, how to not fubar your git history

As with all the git lessons I have learned, this one started with a huge mistake. I had merged a commit to master that was breaking pages I didn’t know existed. The sage developer sitting on my right shoulder told me that since I was unsure of how to fix the problem I should revert the offending commit forthwith.

I had two choices of commit to revert:

remove the offending commit from the branch history
remove the merge commit, where the broken code had infiltrated master

While the former is like a surgical excision of a moment in history (eternal sunshine of a spotless git history?), the latter will preserve your branch history, and the context surrounding the changes made. I like a lot of context, so I chose to revert the merge commit.

When you undo a merge, git doesn’t know how to reassign the branches. Was the left one master? The right? Does git know left from right? Who is on first? Take the guesswork out of the equation

$ git checkout -b revert_branch_name
$ git revert -m 1 <SHA of commit to be reverted>

Warning: If you don’t make the new branch first, you will mess up your merge history on master. Trust me. Just make the branch first. Now you can ensure that your test suite will pass with the commit reverted before you do something silly, like merge even more breakage into master. Crisis averted.

Awesome.

Slack

Mon, 03 Feb 2014 16:06:13 GMT

Slack: Getting Past Burnout, Busywork and the Myth of Total Efficiency, by Tom DeMarco

Trimming the slack from an organization in an effort to reduce overhead will leave a company with an increased profit margin at the cost of becoming inflexible. “Slack is the natural enemy of efficiency, and efficiency is the natural enemy of slack.” Knowledge work cannot be optimized in the same manner as manufacturing because it is creative by nature, and creative work is requires immersion, experimentation, and… well.. slack.

a number is a double is a float

Mon, 03 Feb 2014 03:56:17 GMT

JavaScript has a single number type — a 64-bit floating point number. Since there is no separate integer type, 1 and 1.0 are the same value.

One of the consequences is that the / operator may return a floating point number even if both operands are integers. Goodbye casting!

JavaScript: The Good Parts

Awesome.

git add -p

Sun, 02 Feb 2014 04:51:09 GMT

I learned a new tool that has quickly become an essential part of my workflow. git add -p lets you add small hunks of modified code to the index from your working tree in order to prepare your next commit.

The best refactors are small atomic changes, and while I may not always work incrementally, I still want my git history to reflect the ideal. I use git add -p before every commit to ensure that each code change is intentional.

$ git add -p
diff --git a/send.rb b/send.rb
index 5e89eaf..a0f25b6 100644
--- a/send.rb
+++ b/send.rb
@@ -17,5 +17,5 @@ queue = channel.queue("hello")
 channel.default_exchange.publish("Hello world!", :routing_key => q.name)
 puts " [x] Sent 'Hello World!'"

-# close the connection
-conn.close
\ No newline at end of file
+#close the connection
+connection.close
Stage this hunk [y,n,q,a,d,/,e,?]?

Woah — that is a lot of information… lets go through by the numbers:

the initial command
tells me that it is showing the changes made in the send.rb file. ‘a’ is the version already in the commit history, and ‘b’ is the copy in the working directory.
shows the commit SHA range for this branch
tells me that lines marked with a ’-’ are from the committed version of send.rb
tells me that lines marked with a ’+’ are from the working directory version of send.rb
@@ -17,5 +17,5 @@ indicates that the hunk under scrutiny begins at line 17 of the committed version (‘-17’) and affects 5 lines. It corresponds to the working directory version starting at line 17 (‘+17’), also affecting 5 lines.
the remaining lines contain the details of the changes in the hunk.

Know your options

Now that you have a hunk, what can you do with it?

Type ’?’ to learn what you can do with your hunk

...
Stage this hunk [y,n,q,a,d,/,e,?] ?
y - stage this hunk
n - do not stage this hunk
q - quit, do not stage this hunk nor any of the remaining ones
a - stage this and all the remaining hunks in the file
d - do not stage this hunk nor any of the remaining hunks in the file
g - select a hunk to go to
/ - search for a hunk matching the given regex
j - leave this hunk undecided, see next undecided hunk
J - leave this hunk undecided, see next hunk
k - leave this hunk undecided, see previous undecided hunk
K - leave this hunk undecided, see previous hunk
s - split the current hunk into smaller hunks
e - manually edit the current hunk
? - print help

The first few options are obvious choices — y (“Yes, please!”), and n (“What was I thinking there?”), as well as q (“Hey, Jane! Get me off this crazy thing!”).

I haven’t quite got a handle on the middle options, but I have skimmed down and used s to split the hunk into smaller pieces. Split will only divide a hunk if there is a logical place for separation, like a gap between changes. For real index-finesse and line-by-line control you have to… edit the hunk (gasp).

Seasoned veterans may blanch at the thought of editing a hunk without a GUI, but I have discovered the secret to command-line-hunk-editing-prowess. The secret is:

Read the instructions

Revisiting the earlier example, I choose e to edit my hunk. I think the original comment on line 10 is better than the new one on line 13 (spacy == better). Selecting e opens your editor of choice with the now familiar hunk, along with…

Lo, and behold! Instructions after the hunk!

Simple enough:

-# close the connection becomes # close the connection — mind the space!
remove +#close the connection entirely
save and close.

Success? If you have done everything right, the hunk will be staged and git will move on to the next hunk in your working directory. If you have made an error you will see the following message:

Your edited hunk does not apply.
Edit again (saying "no" discards!) [y/n]?

Aw nuts. y will let you amend your transgressions (did you remember to save? did you forget to leave a space in place of the ’-’?), and n will abandon your changes and take you back to the drawing board, where you are free to transgress anew.

Awesome.

git retrieve: a drama

Fri, 24 Jan 2014 04:47:46 GMT

Have you ever deleted a file on impulse, only to wake up with remorse later? It doesn’t matter if later is the next morning or the next week, thanks to git you have the power to reanimate gone-but-not-forgotten files. Nothing is ~~ever~~ EVER lost with git. So how do you retrieve a file that has been deleted in a previous commit?

stackoverflow has a few suggestions, and I like this one the best:

Git Drama in Three Acts

Act 1 Scene 1: foo.bar Folly

$ cd ~/very_important_project/
$ touch readme.md index.html foo.bar
$ git commit -am "These changes are AWESOME."
[master (root-commit) 9155e0b] These changes are AWESOME.
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 foo.bar
 create mode 100644 index.html
 create mode 100644 readme.md

but wait! foo.bar?! that is just crazy talk?

Act 1 Scene 2: Regret

$ git rm foo.bar
rm 'foo.bar'
$ git add -u
$ git commit -m "I have made a huge mistake."
[master b045a52] I have made a huge mistake.
 0 files changed, 0 insertions(+), 0 deletions(-)
 delete mode 100644 foo.bar

Everything seems back to normal! Act 2 Scene 1: I have made good life-choices

$ echo "#I rock and I am smug about it!" > bar.rb
$ git commit -am "This work is the BOMB."
[master 3535f28] This work is the BOMB.
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 bar.rb
$ cat foo.bar
cat: foo.bar: No such file or directory you heartless creature!

but wait! there is no bar without a foo! where is my foo.bar? How can I find it? I have to find the commit where it was deleted, but how? What if I list out the commits that include a deleted file?

Act 2 Scene 2: Regret

$ git log --diff-filter=D --summary
commit b045a528b4a0824f63562cad867b264983b32c7c
Author: keighty <keighty.leonard@gmail.com>
Date:   Wed Jan 22 20:46:16 2014 -0800

    I have made a huge mistake.

 delete mode 100644 foo.bar

There you are! in commit b045a528. The summary option shows the details, so you can see what files were deleted, when, and by whom. In some cases, there may be more than one change in a commit, so using commitSHA~1 <filename> will pluck the one precious deleted file from the commit and restore it to your working tree.

Act 3: foo.bar’s Return

$ git checkout b045a528~1 foo.bar
$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
# new file:   foo.bar
#
$ git commit -am "Together at last, together forever."
[master 8492eac] Together at last, together forever.
 0 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 foo.bar

EXEUNT.

mongod maintenance

Wed, 22 Jan 2014 15:23:10 GMT

My laptop’s battery died, and I let it. Not a big deal, it is an old laptop and is used to being abused. Every time it does a hard shut down like that I wonder if it will recover. The laptop recovered just fine, but mongo did not. When my computer coughed back to life, I politely requested a meteor server, to which meteor replied that it could not start my server because it could not find a running mongodb. My first instinct, naturally, was to start it up again:

$ mongo start
MongoDB shell version: 2.4.6
connecting to: start
Mon Jan 20 07:42:47.289 Error: couldn't connect to server 127.0.0.1:27017 at src/mongo/shell/mongo.js:145
exception: connect failed

Followed by a trip to the logs:

$ tail /var/log/mongodb/mongodb.log
*************
[initandlisten] exception in initAndListen: 10309 Unable to create/open lock file: /var/lib/mongodb/mongod.lock errno:13 Permission denied Is a mongod instance already running?, terminating
dbexit:
[initandlisten] shutdown: going to close listening sockets...
[initandlisten] shutdown: going to flush diaglog...
[initandlisten] shutdown: going to close sockets...
[initandlisten] shutdown: waiting for fs preallocator...
[initandlisten] shutdown: closing all files...
[initandlisten] closeAllFiles() finished
[initandlisten] shutdown: removing fs lock...
[initandlisten] couldn't remove fs lock errno:9 Bad file descriptor
dbexit: really exiting now

A lock file, you say? Well! I know what to do with those!

$ rm /var/lib/mongodb/mongod.lock
rm: remove write-protected regular file `/var/lib/mongodb/mongod.lock'? n

No, I am not sure — thanks for asking! After a deep breath and a reminder that I should not be so hasty (rtfm, katie), I found this note on mongodb.org:

Note: Because mongod rewrites all of the database files during the repair routine, if you do not run repair under the same user account as mongod usually runs, you will need to run chown on your database files to correct the permissions before starting mongod again.

A file rewrite will include a lock file, will it not?

$ sudo -u mongodb mongod --repair --dbpath /var/lib/mongodb/
[sudo] password for keighty:
...
sudo service mongodb start

Awesome.

embrace HRT for collabrative nirvana

Mon, 20 Jan 2014 15:07:15 GMT

Team Geek: A Software Developer’s Guide to Working Well with Others by Brian W. Fitzpatrick and Ben Collins-Sussman.

With decades of collective experience, these two authors distill the essential philosophy required to be part of a healthy development team: HRT — humility, respect, and trust. These values are covered in the first chapter and the remainder of the book describes how to best use them as an engineer and as a manager. Some techniques particularly resonated with my world view.

Fail early. Fail fast. Fail often

It is easy to pay lip-service to this philosophy, but it is impossible to adopt without a culture of HRT. From my own limited experience I can add — Communicate early. Communicate fast. Communicate often. — as another mantra of healthy team dynamics. Failure is always an option, but without HRT and open lines of communication it can take much more time to discover.

Make time vs manager time

Fitzpatrick and Collins-Sussman drew on a lot of great resources, and there is as much gold in the footnotes as there is in the text. They referenced Paul Graham’s essay, Maker’s Schedule, in which he contrasts the different working styles of managers and makers (writers or programmers). The manager’s schedule can be divided into hour-long chunks that may be apportioned to meetings, appointments, or tasks.: the essence of David Allen’s GTD (Getting Things Done). Makers, on the other hand, cannot be effective in one hour intervals. Graham writes that a single meeting in the afternoon can affect the productivity for a whole day. First it breaks up the afternoon into two pieces that are not long enough to really get into the zone, then it psychologically leaks into the morning where you are “less likely to start something ambitious”. I can definitely identify with this propensity. The cost of task switching around meetings often results in a lost day.

Core values

HRT is the heart of high-functioning teams of any ilk. When I was interviewing for my first job in software development (all those weeks ago) the most difficult question an interviewer could pose to me was “where do you see yourself in 5 years”? It was difficult because I had just spent the last five years dreaming about writing code for a living. I couldn’t even verbalize what I was looking for in a company, only that I would know it when I met it. Now I can identify the traits I was looking for: teams that value writing code and shipping product, consensus based team decision making, asynchronous communication (I am not room-meat), and of course, they’ve got to have HeaRT.

Awesome.

open-source setup:

Sat, 18 Jan 2014 19:18:29 GMT

`git checkout` the Ego.

In the interests of giving back to the software community, I have been looking for an active open-source project to contribute to. I am working to gain mastery of ruby on rails, so naturally I gravitate towards projects of that ilk. Unfortunately, I have gained just enough knowledge to be a little smug, so when I found a project I wanted to work on I scrolled immediately to the README and started scanning. The first block of text was headed by:

Start here if this is your first ruby on rails project…

…with a link to a blog post. I kept scanning for information that would be relevant to ME. This was not my first ruby on rails project. I have my elaborate dev environment set up just the way I like it, thank you very much, and I can reproduce your project with a flick of my config file, I am sure (chuckles condescendingly). This is a snippet of the proceeding workflow:

Workflow of the egotistical programmer.

$ git clone repo
$ cd repo/
$ bundle install
Errors: need to update your rubygems
$ gem update --system
$ bundle exec rake db:setup
Errors: need postgres installed (no inkling in the README)
$ brew install postgres
... follow half of the required instructions
Errors
Frustration because you don't know how to follow instructions
$ brew info postgres
...
"ooooohhhhhhhh... I missed a step"
$ follow instructions this time
$ bundle exec rake db:setup
Errors: need redis installed
$ brew install redis
$ follow instructions because I have learned my lesson
$ bundle exec rake db:setup
Errors: redis config file missing information
Frustration because redis config file was removed 100 commits ago
...
Determine that first pull request
will be an amendment to setup documentation
Rebuild config file and add host name
$ bundle exec rake db:setup
Errors: required seeds file was removed 100 commits ago
$ bundle exec rake db:migrate
$ SUCCESS!!!
$ bundle exec rake spec
Errors: run db:test:prepare
$ bundle exec rake db:test:prepare
Errors
Frustration
$ RAILS_ENV=test bundle exec rake db:migrate
$ SUCCESS!!
$ bundle exec rake spec
Errors... sigh.. more errors
$ limp away for lunch, live to contribute another day

Getting a new project setup and functional is not an easy chore, and I was quite chastened when I swallowed my pride and checked out the blog post for first timers, which boiled down to:

Install VirtualBox
Install Vagrant
git clone repo
cd repo
vagrant up
vagrant ssh
cd /vagrant
bundle install
bundle exec rake db:migrate
bundle exec rails s

I could have been contributing hours ago, when I still had momentum! Perhaps my first pull request should still be to modify the README. I would rephrase the heading to read “For all developers: get started right here”, and then remove every other instruction that followed.

Perhaps I should go get started…

Awesome.

Meteor user

Wed, 15 Jan 2014 03:38:16 GMT

“Getting Started with Meteor.js JavaScript Framework” by Isaac Strack is a great introduction to a powerful tool.

User accounts in three easy steps

Meteor rolls its own user accounts system, and getting it running on your application is a three step process:

command line: Add the relevant packages

$ meteor add accounts-base
accounts-base: A user account system
$ meteor add accounts-password
accounts-password: Password support for accounts
$ meteor add email
email: Send email messages
$ meteor add accounts-ui
accounts-ui: Simple templates to add login widgets to an app

html: Add a div to hold the login button

<div style="float:right; margin-right:20px;">
{{loginButtons align="right"}}
</div>

client javascript: Add an Accounts config

Accounts.ui.config({
passwordSignupFields: 'USERNAME_AND_OPTIONAL_EMAIL'
});

The catch

While the server side and client side code looks similar they are definitely not interchangeable. Server code run on the client side will likely fail silently, but client code run on the server will crash the app. One place where I got tripped up was accessing the userId:

Server side `this.userId`

In Meteor server side javascript the context is clear: this is called in the context of a Meteor object, and you can therefore make inquiries about its properties. {“Use this.userId inside the server publish function”} to access the id of the current user (or null if no user is logged in — from the docs).

Client side: `Meteor.userId()`

In the Meteor client side, this has a different context: the window object. The window object has a userId property that can be set at Document.ready(), but in the case that a user logs out and another user logs in to that same window without a page refresh, window.userId will retain the id of the first user. {“Use Meteor.userId() anywhere BUT in the server publish function”} to get the current user id (or null if no user is logged in — from the docs).

Thanks to this stackoverflow response, the Meteor docs, and packtpub’s awesome support page for helping me get reacquainted with javascript contexts.

Awesome

get clean

Tue, 14 Jan 2014 05:29:03 GMT

I experiment a lot before I choose an implementation I like. I am sure I am not alone in making a few (or many) false starts, and yet feeling sick at the thought of deleting any code. What if one of those paths I started down was the right one? This loss aversion often leads to working trees of non-working code, and git stash-es that I will never revisit. If I stop myself from going too far down yet another road without an end in sight, attempts to clean up my working directory often stall miserably at the end of a git checkout .:

On branch intelligent_branch_name
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
# ../test/yay_I_wrote_tests.rb
# ../helpers/yay_I_wrote_a_helper.rb
# ../helpers/how_many_helpers_did_I_write.rb
# ../views/i_dont_remember_creating_this_file.rb
# ../models/oh_thats_where_that_went.rb

Once upon a time, I would click my way through rm after rm. A small improvement in rm efficiency came when I discovered I could rm more than one file at a time (don’t judge me, I am new at this). Still, the biggest break-through came when a coworker, perhaps observing my struggles, pointed me to git-cheatsheet — a interactive visual tool to display the relevant commands for each stage of my git workflow.

Enter `git clean`

Git clean is a force to be reckoned with, and not just because it doesn’t do anything unless you force it.

Use the -n flag to describe the action the cleaner would take

$ git clean -n
Would remove ../helpers/yay_I_wrote_a_helper.rb
Would remove ../helpers/how_many_helpers_did_I_write.rb
Would remove ../views/i_dont_remember_creating_this_file.rb
Would remove ../models/oh_thats_where_that_went.rb
$

Use the -f flag to force the cleanup

$ git clean -f
removed ../helpers/yay_I_wrote_a_helper.rb
removed ../helpers/how_many_helpers_did_I_write.rb
removed ../views/i_dont_remember_creating_this_file.rb
removed ../models/oh_thats_where_that_went.rb
$

Now I can be as crufty as I want to be in my working tree, because git will be there to clean up the mess.

Awesome.

Factoring In Nests

Wed, 30 Oct 2013 05:29:03 GMT

I am building an application that uses several nested attributes: Users have many Goals (Goals belongto Users), and Goals have many Hours (Hours belongto Goals).

Using FactoryGirl to create a User is straightforward:

FactoryGirl.define do
  factory :user do
    sequence :name  { |n| "person#{n}" }
    sequence :email { |n| "person#{n}@example.com" }
  end
end

describe User do
  before  { @user = FactoryGirl.create(:user) }
  subject { @user }
  it { should respond_to(:name) }
  ...
end

How do I create Goals that are linked to the User?

How to create Factories for nested attributes

1. Make a new Factory for producing Goals:

FactoryGirl.define do
  factory :goal do
    sequence :description { |n| "my goal #{n}" }
    sequence :motivation  { |n| "my motivation #{n}" }
  end
end

2. Call the Goal Factory from the User Factory and store the Goals in an Array

FactoryGirl.define do
  factory :user do
    sequence :name { |n| "person#{n}" }
    sequence :email { |n| "person#{n}@example.com" }

    goals { Array.new(1) { FactoryGirl.build(:goal) } }

  end
end

Your objects will look like this:

> p @user
#<User id: 1, name: "person1", email: "person1@example.com", ... >
> p @user.goals.first
#<ActiveRecord::Associations::CollectionProxy [#<Goal id: 1, user_id: 1, description: "my goal 1", motivation: "my motivation 1", ...]>

Create Hours for Goals in the same manner: create a Factory for making Hours, call the Hours Factory from the Goals Factory.

FactoryGirl.define do
  factory :hour do
    duration { 100 * rand(36) }
    ...
  end
end

FactoryGirl.define do
  factory :goal do
    sequence :description { |n| "my goal #{n}" }
    sequence :motivation  { |n| "my motivation #{n}" }

    hours { Array.new(3) { FactoryGirl.build(:hour) } }
  end
end

Checking your objects:

> p @user
#<User id: 1, name: "person1", email: "person1@example.com", ... >
> p @user.goals.first
#<ActiveRecord::Associations::CollectionProxy [#<Goal id: 1, user_id: 1, description: "my goal 1", motivation: "my motivation 1", ...]>
> p @user.goals.first.hours.first
#<ActiveRecord::Associations::CollectionProxy [#<Hour id: 1, goal_id: 1, duration: 3200 , ...]>

Awesome.

Meatier Apps With Meteor

Mon, 21 Oct 2013 05:29:03 GMT

Meteor is a framework that allows you to write web applications using only javascript. All the work is done on the client-side. Meteor embraces new principles of web design: data on the wire, one language is enough, and simplicity makes for productivity.

Simplicity is something I value as well, and while setting up meteor on my linux machine was not a three-step process as outlined in their examples, it only took a little tinkering to figure out the problem. First, a look at the three meteor setup steps:

Install

$ curl https://install.meteor.com | /bin/sh

Create a sample application

$ meteor create testapp

Run the sample application

$ cd testapp
$ meteor

This is where the meteor server is supposed to start, connecting to your mongo installation, and serving your application on port 3000:

$ meteor
[[[[[ /path/to/testapp ]]]]]

=> Meteor server running on: http://localhost:3000/

Instead, my terminal became quite upset about it:

[[[[[ /path/to/testapp ]]]]]

Unexpected mongo exit code 1. Restarting.
Unexpected mongo exit code 1. Restarting.
Unexpected mongo exit code 1. Restarting.
Can't start mongod

Whups! Everything seems normal in my mongo installation, but after a lot of searching I finally found the problem on stackoverflow. From among the answers I found this solution:

I’m also getting this error on Ubuntu. As mentioned, it’s caused by mongo and mongod from ~/.meteor/tools/latest/mongodb/bin being compiled with an older version of glib. You can replace the version of mongo bundled by meteor with the version installed in your system:

cd ~/.meteor/tools/latest/mongodb/bin/
mv mongo mongo-backup
mv mongod mongod-backup
ln -s /usr/bin/mongo
ln -s /usr/bin/mongod

Brilliant! Following these instructions, I was finally able to start meteor.

Getting down to business

Now to test out some of the awesome functionality of this framework. I followed an example and created a todo list:

$ meteor create --example todos
$ cd todos
$ meteor deploy totally-fake-url.meteor.com
Deploying to totally-fake-url.meteor.com.  Bundling...
Uploading...
Now serving at totally-fake-url.meteor.com

Playing around with creating new todos and lists, it is clear that data is being added and changed instantly and persistently.

A Meteor application is a mix of JavaScript that runs inside a client web browser, JavaScript that runs on the Meteor server inside a Node.js container, and all the supporting HTML fragments, CSS rules, and static assets.

I am really looking forward to diving further into the docs to learn more about this incredible framework. AWESOME

Partial to Ajax

Sat, 19 Oct 2013 05:29:03 GMT

I have been building a new project and getting creative with views. I am working with bootstrap and jquery to develop tabbed displays, and in the process have been gleaning a deeper understanding of rails routes, controllers, and assets. Using Ajax to load partials into rails views is a three part process:

1. In the View, Add a Link

/path/to/view.html.erb

<%= link_to "Show my partial", path_to_controller, remote: true %>

The key difference in the syntax for this link is “remote: true”, which signals to the controller action to respond using ajax, not html.

2. Create a Partial

/path/to/view/mypartial.html.erb

<div>
  <%= form_for @model do |f| %>
    <%= f.label :my_text_field, "Text Field Label" %>
    <%= f.text_area :my_text_field %>
    <%= f.submit "New", class: "btn btn-primary" %>
  <% end %>
</div>

3. In the Controller Action, Add a js Response

/path/to/controller.rb

def my_action
  respond_to do |format|
    format.js
  end
end

Normally, Rails will render the view that corresponds to the name of the action in response to an html request. The format.js line asks rails to look for a javascript file instead of an html file when it tries to render a view.

3. Create a my_action.js.erb File to Shape the Ajax Response

/path/to/view/my_action.js.erb

$("#your-placeholder-id").prepend('<%= escape_javascript(render "path/to/view/my_partial"') %>

Files with multiple extensions are processed from last to the first one. With js.erb, the erb will process all the ruby content first, and then the javascript will be run. Your js.erb file should be saved in the same folder as the rest of your controller’s views. This is an example :

app/views/users/
├── _my_partial.html.erb
├── edit.js.erb
├── show.html.erb

Voila! When you click the link in your view the partial is loaded using Ajax.

Awesome.

Rails, You Can Depend on Javascript

Tue, 17 Sep 2013 05:29:03 GMT

I am between computers at the moment, and I really really miss my OSX development environment. Dependency installation is so smooth and painless on mac, that I had quite forgotten the complications of setting up a complete environment in linux. For instance

Rails requires a javascript runtime environment

I am running an Ubuntu 9.4 remix on my little Atom Netbook. Getting rails up and running anew is a lesson in returning to fundamentals.

$ gem install rails
$ rails new testapp -T
$ rails generate rspec:install
...
Could not find a JavaScript runtime. See https://github.com/sstephenson/execjs for a list of available runtimes

Why does rails use a javascript runtime?

A web application does not exist in a ruby/rails vacuum — rendering an html page requires css and javascript as well. By default, the ‘rails new’ command generates a Gemfile containing a few suggested goodies, including uglifier, which compresses javascript assets. Dealing with multiple layers of javascripts can hurt application performance, which is why rails has adopted a compression strategy. Uglifier minifies your javascript by removing all the whitespace.

From the RailsGuides:

You will need an ExecJS supported runtime in order to use uglifier. If you are using Mac OS X or Windows you have a JavaScript runtime installed in your operating system. Check the ExecJS documentation for information on all of the supported JavaScript runtimes.

Setup Node.js on Linux

In Ubuntu 10.4 and above, a JavaScript runtime is included, but installing Node on an older Ubuntu distro is not as simple as

$ sudo apt-get nodejs

I found the key sequence of commands on the Node.js website. First, update your packages and install node.js dependencies: python, g++, make, etc

$ sudo apt-get update
$ sudo apt-get install python-software-properties python g++ make

Next, add the location of the nodejs repository, update, and install.

$ sudo add-apt-repository ppa:chris-lea/node.js
$ sudo apt-get update
$ sudo apt-get install nodejs

Wow — with all of that done, I can finally:

$ rails generate rspec:install
      create  .rspec
      create  spec
      create  spec/spec_helper.rb

Awesome.

Ruby Execution

Wed, 11 Sep 2013 05:29:03 GMT

I have started including an executable in my project setup. Many of the code challenges I have been practicing lately have included file I/O, and while TDD and code exercising with RSpec is still my main process, developing a stand-alone fully-functional project requires something more.

Blocks Code Challenge

I found the blocks code challenge on the UVa code competition website. To sum up, you have to write a program that will parse and carry out a series of commands for stacking blocks.

move a onto b
move a over b
pile a onto b
pile a over b
quit

The commands will be provided in an input file, and an output is specified. Fun problem, right? So, I start my project folder using my thor task, and work awhile adding logic and tests and data as appropriate. My file tree finishes like this:

blocks
├── data
│   ├── input.txt
│   └── output.txt
├── lib
│   └── blocks.rb
├── spec
│   ├── blocks_spec.rb
│   └── spec_helper.rb
├── Gemfile
└── README.md

So, in order to use my script, I would have to call it from the command line:

$ ruby lib/blocks.rb data/input.txt
#=> or
$ data/input.txt | ruby lib/blocks.rb

This is an extremely verbose way to deliver a final product, and I would much rather call $ blocks <file_input>. It turns out that making an executable is easy and elegant, just like everything else in ruby. You just have to declare the ruby environment, include a few notes on usage, load the file tree, and call the class:

#!/usr/bin/env ruby
# blocks
# 10-Sep-2013
#
# Usage:
# ./blocks data/input.txt
#
$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), 'lib'))
require 'blocks'
Blocks.new.process_input

Placing this code in a non-extension file, like ‘blocks’, in the main directory, I can make it an executable by changing the file permissions:

$ chmod +x ./blocks

And I can call it like any other executable:

$ ./blocks data/input.txt

I liked this solution so much, I added it to my thor project setup! Oh, and if you are interested in seeing my solution for the blocks problem, checkout my github repo

Awesome.

Thor Sets Up A Project

Mon, 09 Sep 2013 05:29:03 GMT

A few weeks ago I posted about using Thor to generate system-wide notes files with a standard format. Since then, I have been doing a lot of smaller code challenges, and I wanted to set up a Thor task for generating a standard project file tree.

My Goal Tree:

project_name
├── lib
│   └── project_name.rb
├── spec
│   ├── project_name_spec.rb
│   └── spec_helper.rb
├── Gemfile
└── README.md

I always begin with these files and this organization. A Gemfile and README are essential for setting up the environment and explaining the gist of the project. I am most comfortable with RSpec, and to get into the practice of TDD, setting up a testing environment right away is non-negotiable. All of these standard files contain some automatic content as well:

spec_helper.rb loads the library

$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
require 'rspec'
require 'project_name'

projectnamespec.rb sets up the first test

require 'spec_helper'

describe ProjectName do
  before (:each) {  }
  subject {  }

  it "should pass"
end

Gemfile specifies the right gems for setting up the project

ruby '2.0.0'
gem 'rspec'

There are other files that could have auto-generated content as well, like a Rakefile, or a config.ru, but this is a good start.

Create a Thor task (rb_project.thor):

#!/usr/bin/env ruby
require "rubygems"
require "thor"

class RbProject < Thor
  desc "init", "creates a ruby project with rspec"

  def init(title="new_project")
    Dir.mkdir(title)
    Dir.chdir(title)
    Dir.mkdir("spec")
    Dir.mkdir("lib")
    filename = "#{title}"

    open(File.new("README.md", "w"), "w") do |note|
      note.puts "# #{title}"
    end

    open(File.new("Gemfile", "w"), "w") do |my_gem|
      my_gem.puts "ruby '2.0.0'"
      my_gem.puts "gem 'rspec'"
    end

    open(File.new("spec/spec_helper.rb", "w"), "w") do |spec|
      spec.puts "$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))"
      spec.puts "require 'rspec'"
      spec.puts "require '#{filename}'"
    end

    open(File.new("spec/#{filename}_spec.rb", "w"), "w") do |spec|
      spec.puts "require 'spec_helper'"
      spec.puts ""
      spec.puts "describe #{camelize(filename)} do"
      spec.puts "  before (:each) {  }"
      spec.puts "  xit 'should pass'"
      spec.puts "end"
    end
  end

  private
    def camelize(snake)
      title = snake.split('_').each do |word|
        word.capitalize!
      end
      title.join('')
    end
end

I created a private method camelize in order to change the snake_case project name into a CamelCase class name.

As before, I install my thor task system wide:

$ thor install rb_project.thor

And now my project generation task can be run from any directory, anywhere on my system:

$ thor list
note
----
thor note:create  # creates a notes file in markdown

rb_project
----------
thor rb_project:init  # creates a ruby project with rspec

$ thor rb_project:init test_project
$ tree test_project/
test_project/
├── lib
│   └── test_project.rb
├── spec
│   ├── spec_helper.rb
│   └── test_project_spec.rb
├── Gemfile
├── README.md

2 directories, 6 files

Awesome.

ARGF all you want

Fri, 06 Sep 2013 05:29:03 GMT

Many of the fun, junior code challenges I have encountered deal a lot with input/output of data, and I found a great ruby feature that provides flexibility at the command line.

Data Input

There are a lot of ways to treat data at the command line, but the one I was most familiar with is passing in a filename as an argument:

$ ruby example_script.rb file1.csv file2.csv ...
$ ruby example_script.rb file1.csv | more
$ ruby example_script.rb file1.csv > output.txt

Filename Input

Using ARGV, we can access the list of filenames provided at the command line. ARGV is an array containing all the information that follows the command. For example:

$ example_command apple banana orange
# ARGV = ["apple", "banana", "orange"]

One can process each filename in ARGV using regular array methods:

ARGV.each do |arg|
  process_string(File.read(arg))
end

File.read(arg) reads the file into a “\n” delimited string, which can be processed further. If there are no arguments, ARGV is an empty array.

Piped Input

Say that we want to enter data directly into a program rather than use a fixed filename at the command line — a useful technique for when we want data to flow out of one process and into another:

$ cat file1.csv | ruby example_script.rb
$ cat *.csv | ruby example_script.rb
$ ruby example_script.rb < file1.csv

Using the pipe, ”|” or ”<”, we can push a string of data from one process into another, but what happens to our program when we try to do this?

$ example.csv | ruby example_script.rb
TypeError: can`t convert nil into String

We get a TypeError! Recall that command line arguments are stored in ARGV, but we are not providing any command line arguments. ARGV is an empty array, and we are attempting to open a file from ARGV[0] = nil. Shame on us! How do we access data that is piped in as well as data that is appended to the command line? Do we have to check for the source of the data before we can treat it? Turns out, we DON’T.

The ARGF Solution

Ruby has a nifty interface for handling data input, regardless of whether it arrives as a command line argument or from another data source. We can replace the code we wrote earlier with a single line:

process_string(ARGF.read)

If ARGV is not empty, ARGF will assume it is an array of filenames and will treat them accordingly. If ARGV is empty, it will read from $stdin to get the data passed in via the pipe. One caveat is that you can’t read from both ARGV and the pipe: if ARGV != [ ], $stdin is ignored.

Awesome.

Hammer-out Tasks with Thor

Sat, 31 Aug 2013 05:29:03 GMT

A few months ago I gave a small lightning talk to my code school class about automating tasks using Rake. Before I learned ruby I used bash scripts to accomplish similar things, but was often bogged down in learning the syntax of string-processing essentials like awk or sed. I wished for the simplicity of Ruby, and thought that Rake was the bees-knees. After my talk, Chuck, my PCS mentor, put together a repo of Thor tasks that accomplished the same tasks but could be installed system wide.

Thor is my new hero

Unlike Rake tasks, which are confined to the directory containing the Rakefile, Thor tasks can be installed on your system and called from anywhere. My note files are all formatted in a particular way. Before Thor, I used a rake task:

$ cd notesDirWithRakefile
$ rake post title="new post title"
$ Creating new post: "./130830_new_post_title"

The Rake task auto-formats the filename, replacing spaces with underscores, and prepends today’s date so that they will sort in the order they are made. This worked great for when I wanted to keep notes in that one folder, but later I wanted to be able to keep notes on a particular project. I would either have to copy the Rakefile (two places to update if I made any changes), or create a symbolic link (which would break if I ever moved my Rakefile). Neither of these options seemed to be the most practical. Enter Thor:

Creating the Thor task is as easy as pie:

require 'thor'

class Note < Thor
  desc "create", "creates a notes file in markdown"
  def create(title="new note")
    slug = title.downcase.strip.gsub(' ', '_').gsub(/[^\w-]/, '')
    date = Time.parse(Time.now).strftime('%y%m%d')
    filename = File.join("./", "#{date}_#{slug}.md")

    if File.exist?(filename)
      abort("rake aborted!") if ask("#{filename} already exists. Do you want to overwrite?", ['y', 'n']) == 'n'
    end

    puts "Creating new post: #{filename}"
    open(filename, 'w') do |post|
      post.puts "# #{title.gsub(/-/,' ').capitalize}"
    end
  end
end

Listing all Thor tasks is easy:

$ thor list
note
----
thor note:create  # creates a notes file in markdown

Installing a Thor task for system use is easy:

$ thor install notes.thor
...
Do you wish to continue [y/N]? y
Please specify a name for notes.thor in the system repository [notes.thor]: note
Storing thor file in your system repository

Uninstalling a Thor task is easy:

$ thor uninstall note
Uninstalling note.
Done.

With my Thor notes task installed system-wide, I can create a note page in any directory, for any reason, without copying code.

Awesome.

Serving Some Java

Sun, 25 Aug 2013 05:29:03 GMT

For a recent java coding challenge I was asked, among other things, to make a server. It has been several years since I last attempted it, and I had forgotten how easy it could be.

Server

A server, like any good conversationalist, is a good listener. A server is a program running on a computer that listens at a port, waiting to receive instructions.

public class Server {
  private static String inputLine;

  public static void main(String[] args) {

    private static final int PORT_NUMBER = 63400

    try {
      // Socket variables
      ServerSocket serverSocket = new ServerSocket(PORT_NUMBER);
      Socket clientSocket = serverSocket.accept();

      // IO variables
      InputStreamReader inputReader = new InputStreamReader(clientSocket.getInputStream());
      BufferedReader bReader = new BufferedReader(inputReader);

      while ((inputLine = bReader.readLine()) != null) {
        System.out.println(inputLine);
      }

    } catch (Exception e) {
      System.out.println(e);
    }
  }
}

The server opens a socket on a specified port (in this case, port 63400). A socket is the point at which two-way communication happens between computer programs. The server opens a socket at a port and waits for a client to bind the socket so they can exchange information. It waits for input from the client, with instructions to print out anything the client puts in (line 18). So where is the client?

Client

The client is even simpler than the server: it finds the server at the specified port and connects to the socket.

public class Client {

  private static final int PORT_NUMBER = 63400

  public static void main(String[] args) {

    try {

      // Socket variables
      Socket socket = new Socket("localhost", PORT_NUMBER);
      PrintWriter pWriter = new PrintWriter(socket.getOutputStream(), true);

      pWriter.println("Hello World!");

    } catch (Exception e) {
      System.out.println(e);
    }
  }
}

With a successful connection, it generates an output stream. If you start the server in one console it will look like it isn’t doing anything, but when you start the client in another console you can watch them communicate. The client writes “Hello World” directly into the port where the server is listening (socket.getOutputStream()), and as instructed, the server obediently prints “Hello World” to the console.

Awesome.

Runtime For Your Life

Tue, 20 Aug 2013 05:29:03 GMT

Eh? is coming along as a functioning interpreted language, but we are still missing a runtime. We have a lexer that chops and tags our code string into tokens, and a grammar that contains the rules for order of operations, and a parser that is generated by RACC from the grammar. The parser assembles the tokens into an evaluation tree (AST), but we need a runtime to do the actual evaluation.

Recall from my previous post that the AST consists of nodes, like this one for a method definition:

class DefNode
  def initialize(name, params, body)
    @name = name
    @params = params
    @body = body
  end

  def eval(context)
    context.current_class.can_methods[@name] = CanMethod.new(@params, @body)
  end
end

When a method definition node is evaluated it adds a new entry to the can_methods hash. The name of the method is its key, and the value is an object (CanMethod.new) that contains the details of method evaluation (parameters and the method body). The context is the scope of the evaluation - in the case of a method call, the context is the enclosing class.

Ok, so when does the CanMethod get evaluated? It happens at runtime. A class node is built and all the method definitions are added, but the node tree will remain untouched until until a method is called.

class CanMethod
  def initialize(params, body)
    @params = params
    @body = body
  end

  def call(receiver, arguments)
    @body.eval(Context.new(receiver))
  end
end

When your program calls a method, a portion of the AST is evaluated. The body of the method object is called with the provided parameters in the context of the method receiver. For ruby newbies, the receiver is the object on the left side of the method call — ie. in foo.bar(3), foo is the receiver and bar is called with the parameter of 3 in the context of foo. The runtime will create a new context for the method (recall that ruby is block scoped, meaning that all variables and blocks within the method are confined to that method… mostly).

class Context
  attr_reader :locals, :current_self, :current_class

  @@constants = {}

  def initialize(current_self, current_class=current_self.can_class)
    @locals = {}
    @current_self = current_self
    @current_class = current_class
  end
end

The context contains all the local variables for the scope as well as a reference to what object is self, and which is super. We are getting very close to putting all these pieces together!

Awesome

Parcel Parser

Sat, 17 Aug 2013 05:29:03 GMT

On the last episode, I described how to label chunks of code for processing through a parser. Strings are labeled as ‘STRING’, class definitions as ‘A’ + ‘CONSTANT’, etc.

What is a parser?

A parser separates and analyzes a piece of text according to a set of rules specified by a formal grammar. The analysis is performed by assembling the tokenized code into an Abstract Syntax Tree (AST) - a tree of nodes that represent what the code means to the language. The AST evaluates the nodes in a similar manner to order of operations in math: each token is placed on the evaluation tree, and expressions are evaluated by reducing each branch in order.

The parser itself is can be written by hand, but I am using RACC: an LALR (Look Ahead, Left to right, Reverse) parser written by tenderlove to generate Ruby programs. How do we specify a grammar that RACC will understand?

Each rule is formatted in the following way:

RuleName:
  OtherRule TOKEN AnotherRule    { code to run }
| OtherRule                      { ... }
;

It is similar to an if/else statement that captures all possible expressions beginning with the most specific (ie: an expression with TWO rules matches line 2) to more general (all other single expressions are captured on line 3). When a token matches a rule, the code in the attached code block is run.

The code blocks correspond to instructions on how to treat matching tokens. For example, the grammar should specify what happens when the code contains a class definition, labeled with ‘A’ + ‘CONSTANT’.

...
# Class definition
Class:
  A CONSTANT Block { result = ClassNode.new(val[1], val[2]) }
;
...

When the parser catches a class token it will create a new ClassNode with the class name (CONSTANT) and the block as arguments. The val[] array refers to the grammar rule [A, CONSTANT, Block].

Some tokens are parsed very close to AS IS:

Literal:
  NUMBER { result = LiteralNode.new(val[0]) }
| STRING { result = LiteralNode.new(val[0]) }
| TRUE   { result = LiteralNode.new(true) }
| FALSE  { result = LiteralNode.new(false) }
| NIL    { result = LiteralNode.new(nil) }
;

Each literal triggers the creation of a new LiteralNode with its value as the only argument.

The Nodes are outlined in a Nodes class, where the rules of evaluation are defined.

...
class ClassNode
  def initialize(name, body)
    @name = name
    @body = body
  end

  def eval(context)
    eh_class = CanadianClass.new
    context[@name] = eh_class
    @body.eval(Context.new(eh_class, eh_class))
    eh_class
  end
end
...

A ClassNode is initialized with two params (recall from the grammar: ClassNode.new(val[1], val[2])), the class name and its code block. The context is like scope - it can hold modules, classes, methods, attributes, aliases, requires, and includes. Classes, modules, and files are all Contexts (definition from Rdocs). Evaluation of a ClassNode begins by assigning the class to a context and evaluating the code block, which adds more contexts. In this way, a tree structure is formed - an AST - which does all the interpreting work for the new language.

When the grammar and node definitions are complete, RACC will generate a parser:

$ racc -vo parser.rb grammar.y

The parser is a relatively obtuse set of methods and state transition tables, and when you run code through the parser you get an AST that follows the grammar you have defined.

So, now I have a lexer and a parser, but I still can’t run my code. I need to define a runner class that will put all these parts together, but first, I need a break.

Awesome.

Flex Your Lexer

Fri, 16 Aug 2013 05:29:03 GMT

A Lexer is a ruby class with a single method: tokenize(). Its purpose is to label each chunk of code with a particular token. Strings are labeled as strings, numbers as numbers, and class names as constants. For example,

Lexer.new.tokenize("string")
# =>[[:STRING, "string"]]
Lexer.new.tokenize("True")
# =>[[:CONSTANT, "True"]]
Lexer.new.tokenize("a Car")
# =>[[:A, 'a'], [:CONSTANT, 'Car']])
Lexer.new.tokenize("+")
# =>[["+", "+"]]

“Lexing” is accomplished using regular expressions. The lexer treats the text string of your code like a shish kabob - it slides recognizable chunks off the skewer one at a time and categorizes them (onion, pepper, method definition, etc).

class Lexer
  KEYWORDS = ['a', 'can', 'if', 'else', 'while', 'true', 'false', 'nil']

  def tokenize(code)
    code.chomp!
    i = 0
    tokens = []

    while i < code.size
      chunk = code[i..-1]

      # checks keywords
      if identifier = chunk[/\A(eh\?)/, 1]
        tokens << ['}', '}']
        i += identifier.size

      elsif identifier = chunk[/\A([a-z]\w*)/, 1]
        if KEYWORDS.include?(identifier)
          tokens << [identifier.upcase.to_sym, identifier]
        else
          tokens << [:IDENTIFIER, identifier]
        end
        i += identifier.size
...

In this example lexer I have defined my list of keywords, removed the trailing newline, and begun peeling identifiable chunks off the code example.

If the code chunk matches an ‘eh?’ it will add a closing brace to the token array.
If the code chunk matches a KEYWORD it will be upcased and labeled as a keyword and added to the token array.
If the code chunk matches any word of lowercase letters it will be labeled as an IDENTIFIER and added to the token array.

I continued defining regex rules until I was able to completely tokenize a class definition:

code = <<-EOS
a Canadian
  can curl
    if skip:
      say 'Hurry!'
    eh?
  eh?
eh?
EOS
...
Lexer.new.tokenize(code)
# => [[:A, "a"], [:CONSTANT, "Canadian"], [:CAN, "can"], [:IDENTIFIER, "curl"], [:IF, "if"], [:IDENTIFIER, "skip"], ["{", "{"], [:IDENTIFIER, "say"], [:STRING, "Hurry!"], ["}", "}"], ["}", "}"], ["}", "}"]]

Now that I have an array of tokens, it is time to write my grammar laws that will tell the parser what to do with them.

Awesome

Canadian Flair

Sun, 11 Aug 2013 05:29:03 GMT

For my last month at code school, I have decided to learn what it takes to develop a new programming language. Having been teased for my Canadian accent, I thought it appropriate to incorporate some idiomatic Canadianisms, and call my language Eh? Deciding what actual features I want to implement in my programming language is an interesting task, and I think I have narrowed it down to a couple of standard rules and a couple wacky ones:

Rules for Eh?

blocks of code are delimited by ’:’ and ‘eh?’
classes are declared using “A” keyword
methods are defined using “CAN” keyword
lowercase identifiers are local variables or method names
capitalized identifiers are global variables
no parens for args
last value evaluated in return value
everything is an object

Steps for building an interpreted language
This code has to be provided as input to a lexer.
The lexer will convert that input into tokens.
The parser will organize those tokens into a tree of nodes.
The runtime will evaluate the nodes using ruby.

Prototype

Here is what I would like the final result to look like:

a Canadian
  with toque
  with scarf
  with broom

  can curl
    if skip:
      say "Hurry!"
    eh?
    if lead:
      pass
    eh?
    say "How social the game..."
  eh?

  can say_aboot:
    say "What's it all aboot?"
  eh?
eh?

With this vague action plan and “Create Your Own Programming Language” by Marc-Andre Cournoyer in hand, off we go!

Awesome

Show-off with GitHub and Docco

Wed, 07 Aug 2013 05:29:03 GMT

I was experimenting with some documentation styles today and discovered two bits of gold: gh-pages and docco:

Creating a site for your project using GitHub is a 3 step process:

create a new orphan branch called gh-pages
remove all the contents
fill the branch with your boastful new site

$ cd repository

$ git checkout --orphan gh-pages
# Creates a branch, without any parents
# Switched to a new branch 'gh-pages'

$ git rm -rf .
# Remove all files from the old working tree

From here you can use the GitHub Automatic Page generator to throw up a theme and some standard content, you can roll out a jekyll bootstrap platform with blogging integration, or you can simply:

$ echo "Hello gh-pages!" > index.html
$ git add index.html
$ git commit -a -m "First pages commit"
$ git push origin gh-pages

Add and commit your content to your gh-pages branch, and GitHub will generate what needs to be generated in order to serve your website.

Your pages are hosted for free by GitHub. If you have already configured a personal site using User Pages, your project pages will hang off the end of your domain like this : username.github.io/repo_name

Docco

Docco is a quick-and-dirty documentation generator, written in Literate CoffeeScript. It produces an HTML document that displays your comments intermingled with your code. All prose is passed through Markdown, and code is passed through Highlight.js syntax highlighting.

Docco works with Python, Ruby, JavaScript, Java, and many other languages, with no configuration — just plug and play!

Install Docco with npm:

sudo npm install -g docco

Run it against your code:

docco lib/*.ruby

Docco will create a docs/ folder in your project and will generate an html file for each ruby file in the lib directory. This is where beautifully commented code really pays off, but docco does not format multi-line code the same way. Check out my benchmark.rb for an example of how each comment style is treated by docco.

TADA

State of the QuickUnion

Mon, 05 Aug 2013 05:29:03 GMT

In my last look at the UnionFind algorithm I made a quick-find implementation — finding if two elements were connected was fast but connecting two elements was slow. The union operation is a good candidate for optimization.

Quick Union

One way to optimize the union operation is to reduce the number of array accesses required.

In QuickFind, the sets are arranged in a flat array of elements which all share the same value: they are all wearing the same t-shirt. When we connect elements from different sets, we must make sure that every element in the first set changes its t-shirt to match the elements in the other set. This is a time-consuming process, and uses a lot of t-shirts.

What if only one element had to change its t-shirt? If the set had a single element in charge - a root - then joining elements from different sets would require identifying the t-shirt of the one in charge, and then changing to that element’s t-shirt.

Using an array instead of t-shirts to keep track of the values, we can walk through what this implementation would look like:

Index	1	2	3	4	5	6	7	8	9
Value	1	2	3	4	5	6	7	8	9	\| Each element begins as the root of its own set
union(4,9)	1	2	3	9	5	6	7	8	9	\| root(9) = 9, root(4) = 4, change array[4] = 9
union(2,4)	1	9	3	9	5	6	7	8	9	\| root(4) = 9, root(2) = 2, change array[2] = 9
union(4,7)	1	9	3	9	5	6	7	8	7	\| root(7) = 7, root(4) = 9, change array[9] = 7
find(2,7)	1	9	3	9	5	6	7	8	7	\| root(7) = 7, root(2) = 7. The two are connected

QuickUnion

Again following the basic API:

public class UF
---------------
public UF(int N)
public int find(int p, int q)
public void union(int p, int q)

I adapted the QuickUnion in ruby:

class QuickUnion
  def initialize(size)
    @find_array = Array.new(size){ |index| index }
  end

  # check if p and q have the same root
  def find(p, q)
    verify(p, q)
    root(p) == root(q)
  end

  # set id of p's root to the id of q's root
  def union(p, q)
    verify(p, q)
    @find_array[root(p)] = root(q)
  end

  private
    def root(index)
      # get root until root(index) == index
      if @find_array[index] == index
        return index
      else
        root(@find_array[index])
      end
    end

    def verify(p, q)
      raise NotAnElement if @find_array[p].nil?
      raise NotAnElement if @find_array[q].nil?
    end
end

class NotAnElement < Exception
end

This implementation is a QuickUnion because in an array of 1,000,000 entries, connecting element p to element q is easy: you only have to find the root of p, and then change q to match (one root() and one array access). Finding if p and q are connected takes a little bit longer, because you have to perform two root operations: one for p and one for q, but it is still faster than accessing every element in the array.

Benchmark

$ ruby lib/benchmark.rb
=> Array Size: 100
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.001865)
   0.000000   0.000000   0.000000 (  0.000377)
$ ruby lib/benchmark.rb
=> Array Size: 1000
       user     system      total        real
   0.120000   0.010000   0.130000 (  0.124929)
   0.010000   0.000000   0.010000 (  0.008825)
$ ruby lib/benchmark.rb
=> Array Size: 10000
       user     system      total        real
  10.120000   0.030000  10.150000 ( 10.146124)
   0.520000   0.000000   0.520000 (  0.520377)

Comparing QuickFind (top line) with QuickUnion(bottom line), we can actually measure how well this small optimization of the union operation has increased performance with larger datasets.

Awesome.

State of the UnionFind

Wed, 31 Jul 2013 05:29:03 GMT

I was scraping the rust off my data-structures synapses last week, and found that there was still some functionality underneath. UnionFind is one of those sneaky algorithms that crops up in non-linear arrangements of data, such as computer or social networks. Specifically, it keeps track of sets of connected elements. The API consists of a union(p, q), which connects the two elements p and q, and a find(p, q), which determines if p is in the same set as q.

I learned about this algorithm through Coursera, and their class Algorithms Part 1 with Kevin Wayne and Robert Sedgewick. The focus of this course was not only on implementation of algorithms, but also optimization.

Union Find

A simple model for connectivity is 10 objects numbered 1 through 0:

0 1 2 3 4 5 6 7 8 9

Making random connections will produce disjointed sets of objects:

0 1 { 2 3 9 } { 5 6 } 7 { 4 8 }

A find(p, q) will determine if p and q are in the same set:

find(0, 8) => false
find(2, 9) => true

A union(p, q) will put p and q in the same set:

union(1, 6) => 0 { 2 3 9 } { 1 5 6 } 7 { 4 8 }
union(2, 4) => 0 1 { 5 6 } 7 { 2 3 9 4 8 }

Using an array to keep track of the values, we can walk through what this implementation would look like:

Index	1	2	3	4	5	6	7	8	9
Value	1	2	3	4	5	6	7	8	9
union(4,9)	1	2	3	9	5	6	7	8	9	\| Changes all values that match array[4] to match array[9]
union(2,4)	1	9	3	9	5	6	7	8	9	\| Changes all values that match array[2] to match array[4]
union(4,7)	1	7	3	7	5	6	7	8	7	\| Changes all values that match array[4] to match array[7]

Using the java api provided by Kevin Wayne and Robert Sedgewick:

public class UF
----------------
public UF(int N)
public int find(int p, int q)
public void union(int p, int q)

I adapted the basic union find in ruby:

class QuickFind

  attr_accessor :find_array

  def initialize(size)
    @find_array = Array.new(size){ |index| index }
  end

  def find(p, q)
    location(p) == location(q)
  end

  def union(p, q)
    @find_array = create_union(p, q)
  end

  private

    def location(index)
      raise NotAnElement if @find_array[index].nil?
      return @find_array[index]
    end

    def create_union(p, q)
      target = location(p)
      goal = location(q)
      temp_array = @find_array.map do |element|
        element == target ? goal : element
      end
    end
end

class NotAnElement < Exception
end

Sets are arranged as a flat array of elements which all share the same value: they are all wearing the same t-shirt. When we connect elements from different sets, we must make sure that every element in the first set changes its t-shirt to match the elements in the other set. This is a time-consuming process, and uses a lot of t-shirts.

This implementation is called QuickFind because in an array of 1,000,000 entries, finding if two elements are connected is easy (compare the t-shirt at index p with the t-shirt at index q), but connecting them requires inspection of every element in the array in order (check everyone’s t-shirts) to make sure all the previous connections remain in place.

Benchmark

To compare efficiency of this and future UnionFind implementations, I built a basic benchmark that performs a set number of union and find operations and measures the time to completion:

require 'benchmark'
@size = 10000

def testUnit(object)
  @size.times do
    object.union(rand(@size), rand(@size))
    object.find(rand(@size), rand(@size))
  end
end

Benchmark.bm do |x|
  x.report { testUnit(QuickFind.new(@size)) }
end

Running the benchmark on QuickFind:

$ ruby lib/benchmark.rb
=> Array size = 100
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.002745)
$ ruby lib/benchmark.rb
=> Array size = 1000
       user     system      total        real
   0.140000   0.000000   0.140000 (  0.137957)
$ ruby lib/benchmark.rb
=> Array size = 10000
       user     system      total        real
  13.790000   0.020000  13.810000 ( 13.866528)

You can see from these measurements that increasing the size of the array by a factor of 10 each time comes with a logarithmic increase in processing time. There is definitely room for optimization, so stay tuned for more refinements. In the meantime,

Fun Rubyisms

Array initialization is a snap in ruby! Compare the initialization line in the QuickFind algorithm above with the equivalent java implementation:

@find_array = Array.new(size){ |index| index }

versus

...
int[] arry = new int[size]
for (int i = 0; i < size; i++) {
    id[i] = i;
}

Both result in an array of the specified size, but ruby is slightly less sprawling.

Also, check out lines 27 - 29

temp_array = @find_array.map do |element|
  element == target ? goal : element
end

Using map() eliminates the need for a separate array declaration. It is the equivalent of:

temp_array = []
@find_array.each do |element|
  temp_array << element == target ? goal : element
end
return temp_array

and is certainly more readable than the equivalent in java:

int pid = id[p];
for (int i = 0; i < id.length; i++)
    if (id[i] == pid) id[i] = id[q];

Awesome

Procuring Provisions

Sun, 28 Jul 2013 05:29:03 GMT

I was trying to add a database to my JSGames app — it is a basic sinatra app with a few goodies already in place — and I was having some difficulty deploying it to Heroku. Rails (for the most part) automagically creates and maintains its database, and as long as I remember to move any references to sqlite3 into the development and test environments, I have never had trouble provisioning one on Heroku either. Rolling a web application from scratch with sinatra, however, requires a little more savvy.

My Gemfile had a familiar structure:

source 'https://rubygems.org'
ruby '2.0.0'
...
gem "sinatra"
gem "activerecord"
gem "sinatra-activerecord"
gem "pg"

So far, so familiar. I added config/environments.rb to outline the details of the database I would like provisioned (using postgres locally as well as in production):

require 'active_record'
require 'uri'
...
db = URI.parse(ENV['DATABASE_URL'] || 'postgres://localhost/mydb')
ActiveRecord::Base.establish_connection(
  :adapter  => db.scheme == 'postgres' ? 'postgresql' : db.scheme,
  :host     => db.host,
  :port     => db.port,
  :username => db.user,
  :password => db.password,
  :database => db.path[1..-1],
  :encoding => 'utf8'
)

With everything complete, I created a couple of models, migrated, tested it locally, and committed the changes. Heroku compiles the slug just fine, but when I open the application I get the too familiar “Application Error” page. The logs tell me:

ERROR: Invalid DATABASE_URL

Huh? My url is the one that Heroku suggested! How can it be invalid? It was working fine locally, what is the problem? Here is a sample of my google searches from that frantic afternoon:

ERROR: Invalid DATABASE_URL heroku
deploying sinatra pg heroku
environment variables
heroku config variables
usr-env-compile heroku
heroku can’t connect to server
how to roll back a push to heroku
how to recover from the embarrassment of an offline portfolio project
how to start a llama farm

I finally found the answer (aka read the manual) with Heroku Postgres

First: Check to see if your database has been provisioned

$ heroku addons | grep POSTGRES

I ran this command in my sinatra project and found nothing. NOTHING!

When Heroku told me that it couldn’t find my database, that is because it hadn’t MADE my database. So I…

Second: Create a new database

$ heroku addons:add heroku-postgresql:dev
Adding heroku-postgresql:dev on js-games... done, v42 (free)
Attached as HEROKU_POSTGRESQL_GOLD_URL
Database has been created and is available
 ! This database is empty. If upgrading, you can transfer
 ! data from another database with pgbackups:restore.
Use `heroku addons:docs heroku-postgresql:dev` to view documentation.

YAY! My database has been created and found. Still, nothing wrong with…

Third: Double check your database exists

$ heroku config
=== js-games Config Vars
HEROKU_POSTGRESQL_GOLD_URL: postgres://...@ec2-##-###-###-##.compute-1.amazonaws.com:####

Heroku confesses that my database is stored on AWS, and gives a fairly detailed address (redacted on the grounds of national security).

Fourth: Promote the new database to primary

$ heroku pg:promote HEROKU_POSTGRESQL_GOLD_URL
Promoting HEROKU_POSTGRESQL_GOLD_URL to DATABASE_URL... done

Fifth: Check out the new database

$ heroku pg:info
=== HEROKU_POSTGRESQL_GOLD_URL (DATABASE_URL)
Plan:        Dev
Status:      available
Connections: 1
PG Version:  9.2.4
Created:     2013-07-27 22:46 UTC
Data Size:   6.3 MB
Tables:      0
Rows:        0/10000 (In compliance)
Fork/Follow: Unsupported

TADA

Do one final push to heroku and restart the server:

$ git commit -a -m "fixes the last two hours of disasterous troubleshooting"
$ git push heroku master
...
$ heroku restart

Awesome

Git-Picking

Thu, 25 Jul 2013 05:29:03 GMT

or, How to mine for cherries with Git

When I work on an experimental feature in a project branch, I have often found that I will have a mixture of commits I would like to keep, and commits that I wish had never happened. Using the principle of only committing atomic changes, and enjoying the fruits of your detailed-commit-message labour, there is a way to extract the gold from the ore, so to speak.

Locate the gold

Checkout the branch that contains your experiment, and use git log to get a list of all the commits:

$ git checkout form_feature_branch
$ git log --pretty=oneline
a83f4c3aa5d800f846d075748a79a326e0971f67 reorg gemfile
197964c091905a76f2172ca11fcbd49ccfb83c67 adds blog.db
a793fe00c50a5956c5e9f7be48ac5a9861b1eb95 adds forme form to edit...
8394b37494b63754f42ecaf19244fe6d8a36942f adds gitignore for db ...
c52a6fd3f1706584ce3e47abf49ed627a1e28fd3 adds bundler gem for manag...
1686664c4a98221ea04377bcbcf946e62a9f1cfa removes gems for simple...
034d309a612dcce6f10bb55f0da2e78b2f7c10b1 adds cells to project
...

In formfeaturebranch I was testing out different implementations of form generation in erb views. I tried a couple of different gems and found one that suits my sinatra project: Forme is easy to use with or without models, and has great sinatra integration and documentation. Also along the way, I discovered how to include bundler for managing file dependencies and did some minor refactoring of my Gemfile.

I wanted to keep the gem reorganization, as well as the forme, bundler, and gitignore commits, so I made this list of keeper-commits:

a83f4c3aa5d800f846d075748a79a326e0971f67 reorg gemfile
a793fe00c50a5956c5e9f7be48ac5a9861b1eb95 adds forme form to edit...
c52a6fd3f1706584ce3e47abf49ed627a1e28fd3 adds bundler gem for manag...
8394b37494b63754f42ecaf19244fe6d8a36942f adds gitignore for db ...

Everything else is cruft.

Extract the gold

With this list of golden commits in hand, checkout a fresh branch and cherry-pick the commits you like!

$ git checkout -b form_feature_additions
$ git cherry-pick <commit_identifier>

Do this command for each good commit:

$ git cherry-pick a83f4c3aa5d800f846d075748a79a326e0971f67
[master form_feature_additions] reorg gemfile
 1 file changed, 4 insertions(+), 2 deletions(-)
 changed mode 100644 Gemfile
$ git cherry-pick a793fe00c50a5956c5e9f7be48ac5a9861b1eb95
[master form_feature_additions] adds forme form to edit...
...
$ git cherry-pick c52a6fd3f1706584ce3e47abf49ed627a1e28fd3
[master form_feature_additions] adds bundler gem for manag..
...
$ git cherry-pick 8394b37494b63754f42ecaf19244fe6d8a36942f
[master form_feature_additions] adds gitignore for db ...
...

If you look at your git status you will see

# On branch form_feature_additions
# Your branch is ahead of 'origin/master' by 4 commits.

Profit

Now you can merge this branch with master, or continue fleshing out the feature on a clean branch, keeping only the good stuff and abandoning the bad, making experimentation easy.

AWESOME

Sinatra in a console

Mon, 22 Jul 2013 00:00:00 GMT

Sinatra does not have a default console like rails does. If you are writing in a rails environment and need to test out a helper method, or want to check on the state of a record in the database, dropping down into the rails console is as easy as typing:

$ rails c

How can we achieve the same ability in Sinatra?

Tux

Tux is a ruby gem that generates a sinatra console. It reads the config.ru, allows you to interact with App methods, and provides empty request and response objects to for investigating communication between views and the Sinatra controller.

Add Tux to your Gemfile:

group :development, :test do
  ...
  gem 'tux'
end

Don’t forget to bundle!

At the command line, start up tux and use rack.actions to view all the methods available:

$ tux
Loading development environment (Rack 1.2)
>> rack.actions
=> [:request, :get, :post, :put, :patch, :delete, :options, :head, :follow_redirect!, :header, :set_cookie, :clear_cookies, :authorize, :basic_authorize, :digest_authorize, :last_response, :last_request]
>>

You can access any method from your application as well. Of course, if you are working in Sinatra, you probably want to go as lightweight as possible, which is why you can also use:

Basic irb

It is as simple as using irb and requiring your main app file. You have access to all the dependencies and files that your application does by simply requiring that one file. You can check state, call methods, create objects, whatever you need:

$ irb
> require './app.rb'
 => true
> User.all
D, [2013-07-22T15:47:49.230146 #54081] DEBUG -- :   User Load (0.2ms)  SELECT "users".* FROM "users"
 => #<ActiveRecord::Relation [#<User id: 1, name: "Gordon Shumway", email: "gordon@example.com", created_at: "2013-07-22 22:45:48", updated_at: "2013-07-22 22:45:48">, #<User id: 2, name: "Fanny Eubanks", email: "fanny@example.com", created_at: "2013-07-22 22:45:48", updated_at: "2013-07-22 22:45:48">]>
>

This can be called even more elegantly at the command line:

$ irb -r ./app.rb

or through a Rake task:

desc "run irb console"
task :console, :environment do |t, args|
  ENV['RACK_ENV'] = args[:environment] || 'development'
  exec "irb -r irb/completion -r ./app.rb"
end

Awesome

Get your kicks on route localhost:4567

Wed, 17 Jul 2013 00:00:00 GMT

I am building a code breaking game along the lines of Mastermind and I encountered a block.

The problem

I have a javascript that generates a secret code.
I have a ruby class that compares a guess to the secret code and returns an analysis of the accuracy of the guess.
How do I get the ruby class to process a guess and secret code from the browser?

I learned something about Ajax today.

An ajax call is built up from a few minimum parameters: a url, the HTTP method, the dataType expected, and a callback function that handles the return. You can include data from the user in the form of a data hash:

var the_url = "http://" + window.location.host + "/game/" + guess_string;
  var mark_string = $.ajax({
    type: "POST",
    url: the_url,
    accepts: "application/json",
    dataType: "json",
    data: { 'code' : secret_code },
    complete: function(data){
      output_mark = data['responseText'];
      process_output(secret_code, guess_string, output_mark);
    }
  });

This code snippet is using Ajax to send the secret code and guess to the Sinatra route.

I learned something about Sinatra today.

A route has three ingredients: an HTTP method, a URL matching pattern, and a code block. Typically they look like this:

get '/' do
  "hello world"
end

But they can also look like this:

post '/game/:guess' do
  game.guess(params[:guess])
end
private
  def game
    @game ||= Mastermind::Game.new_with_code(params[:code])
  end

The code block returns information. That information can be anything, from a plain string (first example), or an html web page, to a JSON representation of an object (second example). What the browser does with the information it receives is the browser’s business.

The overall workflow:

The url on my local machine looks like this: ”http://localhost:4567/game/12345” (line 1). The submitted guess is 12345.
Ajax packages the url with the data hash containing the secret code (line 7).
Javascript submits the HTTP POST request to Sinatra which matches the route.
The route creates a Mastermind::Game object with the provided secret code.
The route calls the guess method on the Game object with the submitted guess.
The route returns a json of the method output (in this case it is a string of +/- characters indicating exact number matches(+) or matched numbers in the wrong position(-)).
In the ajax callback function the data from the json object is parsed and passed on for further processing.

I learned something about routes today.

I had a mental block about routes. I thought that building the “/game/:guess” route in Sinatra meant that the user would be constantly navigating between “/mastermind” and “/game/:guess”, and that the game would always be reloading with a new secret. I was thinking about routes the wrong way — when javascript calls the route, it doesn’t reload the page, it just retrieves data! I had conflated routes with navigation in my imagination, and forgot that a route is just a method call containing a request for information. Just because the information returned is often rendered as an html page doesn’t mean that it has to be html. Which leads me to the penultimate point…

I learned something about curl today.

ANYTHING can make an HTTP request. Ok, maybe that is a slight overstatement.

Browsers interpret the information retrieved using HTTP requests, but they are far from the only category of software that can. Turns out, the command-line functions just as well:

$ curl -X POST -H "Accept: application/json" -d "code=12345" js-games.herokuapp.com/12345
+++++$

The command on line 1 will return the same string I am retrieving with $.ajax in javascript (output on line2). Sure it is a fairly plain result — it doesn’t even include a new line character, for pity’s sake — but it does the job of making an HTTP request to a route and using the result.

I learned something about myself today.

I didn’t even realize I was misunderstanding HTTP! It was one of those concepts I had already checked off my mental list of technologies to cover on my quest to become a web developer. This breakthrough has reminded me that reviewing the fundamentals in the context of learning new technologies will bring a deeper understanding of everything.

Awesome

Mass assignment X Gravity

Mon, 15 Jul 2013 00:00:00 GMT

What is Mass Assignment?

Mass assignment is using a ruby-esque shortcut to interact with models:

This is NOT mass assignment.

def create
  u = User.new
  u.first_name = params[:user][:firstname]
  u.last_name = params[:user][:lastname]
  if u.save
    redirect_to :index, flash: { success: "Created!" }
  else
    render :action => 'new'
  end
end

Each param is extracted from the params hash and assigned explicitly. Any params that are not assigned are thrown on the floor.

This IS mass assignment.

def create
  u = User.new(params[:user])
  if u.save
    redirect_to :index, flash: { success: "Created!" }
  else
    render :action => 'new'
  end
end

The programmer assumes that the params include all the data necessary, and only the data necessary. All params in the hash are used to create the new user record.

Why is mass assignment a problem?

The problem with mass assignment is that if you have a sensitive tag in your model, for example usertype (which could be set to admin), a malicious user can add `user[:usertype] = ‘admin’` to your params hash. Your controller will unwittingly include it in the save command, creating a new admin account for the malicious user.

Is there any way to prevent this diabolical schema scheme?

There is! With Rails 4 comes strong params. Strong params live in the controller and tell it explicitly what params can be trusted.

def create
  u = User.new(user_params)
  if u.save
    redirect_to :index, flash: { success: "Created!" }
  else
    render :action => 'new'
  end
end
...
def user_params
  params.require(:user).permit(:name, :email, :password, :password_confirmation)
end

The only parameters our controller will send to the model are those specified on our white-list. If rails detects a user attempting to access a param that is not on the white-list, said user had better be prepared for grave consequences. Their session will be deleted, their attempt will be logged, and Rails will email their mother.

Awesome.

Best Practices - Pull Requests

Thu, 11 Jul 2013 00:00:00 GMT

Learning how to contribute to open source projects has been really exciting. Adding value to existing projects, getting social in the coding community, and working on production-level code has been a steep but rewarding learning curve. One important thing to learn is the etiquette of creating pull requests. I read a great post about what to include in a good commit, but I also needed to know how to make a good pull request:

Branch it

Before making any changes to an open source project, open a new branch. This will allow you to isolate your changes into separate pull requests. For example, I add Feature A, push to master, and create pull request 1. The next day, I add Feature B, push to master, and create pull request 2. If Feature A has not already been reviewed and accepted, Feature B additions will now be encorporated into pull request 1. This is bad practice. Keep each contribution in a separate branch. This will also allow you to keep your current branch up to date with the upstream repository, without affecting your previous work.

One change, one commit

A commit is not just a backup. Like a migration, your commit history should read like a road map of changes you make to get to your current state — and changes you should undo when things go wrong. If your commits change dozens of lines of code, understanding the changes is going to require a deep dive into the code.

Pithy commit messages

Make it easy on your code reviewer by making your commits small enough, and your commit messages detailed enough that they can be understood at a glance.

The workflow, in general:

fork the repo
git clone to local machine
git checkout -b featureA
git pull upstream master
make your changes
git commit -am "adds one addition to feature A"
git push origin featureA
make pull request

Awesome

Partial to Sinatra

Tue, 09 Jul 2013 00:00:00 GMT

Having just discovered the magic of using partials to refactor html in rails, imagine my delight when I found it to be even easier in Sinatra.

A partial is a just a fragment of html (or erb). They can be used for rendering duplicate or dynamic content into otherwise static pages.

I was writing html for my JSGames sinatra application when I caught myself copying a snippet of html onto my clipboard. Why on earth would I duplicate code? I want to include a block of text on both the index and the game page, but I don’t want to have to modify it in two places when I inevitably get around to rewriting it.

The solution?

Factor out the duplicate html into a partial (begin name with _ for identification purposes). For example _gameDescription.erb

<h1>Game Title</h1>
<p>Game description</p>

Add that partial wherever you want to display this snippet of code:

<%= erb :_gameDescription  %>

For example:

<div class="span4 pagination-centered">
  <%= erb :_solitaireDesc  %>
  <a href="/game" class="btn btn-large">Play</a>
</div>

Check out the final product on JSGames.

Awesome

Red, Green, Refactor! A Workflow for Rails

Wed, 03 Jul 2013 00:00:00 GMT

1. Write a failing test

describe "Static About page" do
  it "should have the content 'About Us'" do
    visit '/static_pages/about'
    page.should have_content('About Us')
  end
end

Use the failures to drive development:

2. No Route

Failures:
  1) Static About page should have the content 'About Us'
     Failure/Error: visit '/static_pages/about'
     ActionController::RoutingError:
       No route matches [GET] "/static_pages/about"

Add one to /config/routes.rb:

DemoApp2::Application.routes.draw do
  get "static_pages/about"
  ...
end

3.No Controller

Failures:

  1) Static About page should have the content 'About Us'
     Failure/Error: visit '/static_pages/about'
     AbstractController::ActionNotFound:
       The action 'about' could not be found for StaticPagesController

Add one to /controllers/staticpagescontroller.rb

class StaticPagesController < ApplicationController
  def about
  end
end

4. No Page

Failures:
  1) Static About page should have the content 'About Us'
     Failure/Error: visit '/static_pages/about'
     ActionView::MissingTemplate:
       Missing template static_pages/about, application/about with {:locale=>[:en], :formats=>[:html], :handlers=>[:erb, :builder, :raw, :ruby, :jbuilder, :coffee]}.

Add a view to /static_pages/about.html.erb

<!DOCTYPE html>
  <body>
    <h1>About Us</h1>
  </body>
</html>

No index.html, no cry

Sat, 29 Jun 2013 00:00:00 GMT

Following along with Rails tutorial by Michael Hartl, I built a sample app with a simple database, linking users to microposts in a twitter-like manner. Everything worked well locally, but when it came time to deploy on heroku, I ran across some interesting stumbling blocks:

Rails 4 does not come with a default index.html

There I was, gleefully watching the slug compile on heroku (after several abortive attempts I realized I had not moved the sqlite3 gem into the development environment — heroku doesn’t like that). All seemed fine, I typed

$ heroku open

and what greeted me? A 404 error page! Something has gone horribly wrong! I have been rejected by heroku! I try again from the beginning, ensuring I have followed all the steps.. what could I have missed? Same result. But, Mr. Hartl, said it was easy… what was I missing?

After some digging, I found a heroku tutorial on getting started with the shiny new Rails 4. Deep inside this article, I found the magic words:

“Note: that Rails 4 no longer has a static index page in production, if you’re using a new app, there may be no root page.”

Orly? Are you saying that the 404 error I was getting was actually generated by my properly deployed app? YES! I made some obvious change to the html

<title>You made it!</title>

and redeployed. TADA!

Awesome.

Hoisting with JavaScript

Wed, 26 Jun 2013 00:00:00 GMT

I am really enjoying Test-Driven Development by Christian Johansen

I picked up this little gem this morning as I was learning about functions and the various objects associated with them:

You CANNOT use function declarations in conditionals. Take this example:

if (String.prototype.trim) {
  function trim(str) {
    return str.trim();
  }
else {
  function trim(str){
    return str.replace(/^\s+|\s+$/g, "");
  }
}

What is wrong with that, you might ask? HOISTING! Both functions are hoisted up to the global variable before execution, which means that the second one always overwrites the first one.

This is a better method:

if(!String.prototype.trim) {
  String.prototype.trim = function trim() {
    return this.replace(/^\s+|\s+$/g, "")
  }
}

This way, we are not overwriting any browser-available Trim methods, we are declaring the method on the String.prototype, which means it is available to ALL strings in scope, AND we have named it so that it will be more visible in a stack trace in case something goes wrong.

AWESOME

Javascript has spies

Tue, 25 Jun 2013 00:00:00 GMT

Spies vs Mocks vs Stubs?

We are covering testing frameworks at PCS this week, particularly Jasmine and Sinon.

Jasmine has a similar structure to RSpec:

describe("Class to describe", function() {
  it("should have some behavior". function() {
    //assertions
  });
});

Also similar to RSpec, Jasmine allows you to spy on behavior. Using spies, I can watch a method call and collect information about it.

Want to know if it was called?
```
expect(method).toHaveBeenCalled()
```
Want to know how many times?
```
expect(method.callCount).toEqual(3)
```

Want to know how it was called?

expect(method).toHaveBeenCalledWith(args)

For example:

describe( "Spy on methods", function () {
  it( "should watch calls to console", function () {
    var mySpy = spyOn(console, "log");
    console.log("test1");
    console.log("test2");
    console.log("test3");

    expect(mySpy.callCount).toEqual(3);
  });
});

Tomorrow, I conquer Stubs and Mocks!

jQuery objects vs strings

Wed, 19 Jun 2013 00:00:00 GMT

Object blocker!!

My pair was stuck for the longest time on the following code snip:

$(function(){
  var new_square = '$(<div class="square"></div>)';

  $('body').on('click','.square', function(){
    // var my_square = $(this);
    $(this).toggleClass('blue');

    b = $(new_square)
      .insertAfter(this);
    a = $(new_square)
      .insertBefore(this);
    b.toggleClass('blue');
    a.toggleClass('blue');
  });
});

We expected out output to include three elements — the original square, one inserted after, and one inserted before. The output we were seeing included only two squares: the original one and the square inserted before. If we inverted the insert calls, we would see only one inserted after. We were totally stumped until we printed out the content of new_square:

$('<div class="square"></div>');

We were actually passing a reference to a jQuery OBJECT into the insert methods, so in essence, we were creating the object once and then passing it around to different locations. When we changed new_square from a jQuery object:

var new_square = '$(<div class="square"></div>)';

to a string

var new_square = '<div class="square"></div>';

We saw our third element appear no problem.

Namespaces and jQuery

Tue, 18 Jun 2013 00:00:00 GMT

Namespaces in jQuery getting you down?

Chuck taught me a great trick today — how to avoid namespace conflicts in jQuery:

(function($){
...
})(jQuery);

This function is being passed jQuery as an argument, and is assigning it the local variable $ — allowing you to use the $ and reference the jQuery library.

Awesome.

Blogging with Jekyll Bootstrap

Sat, 15 Jun 2013 00:00:00 GMT

Building a portfolio has been a lot of fun and has required a couple of different technologies. A fellow PCS student has been blogging a long time, and he suggested using Jekyll in combination with github pages. What a fantastic gem! I found an even more awesome one at jekyll-bootstrap.

Now I am enjoying the power of bootstrap along with the ease of jekyll.

To create a post

$ rake post title="Hello World"

To create a new page:

$ rake page name="pages/about"

Couldn’t be simpler for a beginning blogger.

I admit it, I love TDD

Fri, 14 Jun 2013 00:00:00 GMT

This week @PDXcodeschool we jumped into TDD using minitest/autorun. I have used RSpec in the past, and loved the way it let me manage the logic of my project. I love that ruby has a built in framework — so easy to use!

Use specs to rough out project logic

describe "Car" do
it 'should have a color'
it 'should have a make'
it 'should have a unique serial number'
it 'should get a new serial number when it gets stolen'
it 'should accept passengers'
it 'should not accept more than 3 passengers'
end

All of these tests will be skipped until they are followed by a do…end block.

Take one test at a time and write the code to make it pass
```
describe "Car" do
it 'should have a color' do
car = Car.new("black")
car.color.must_equal "black"
end
...
end
```
If your tests exercise the project well enough, you end up with really tight code. More of a lasagna noodle than spaghetti!

A Flash of Understanding

Sat, 13 Apr 2013 00:00:00 GMT

My page wouldn’t flash the user.

Trundling along the Hartl railstutorial, I needed to write some logic to allow a user to update their information.

As a loyal TDD-er, I wrote a test to check that users see a success message if they are able to update their infomation:

describe "updating with valid information" do
  let(:new_name) { "New Name" }
  let(:new_email) { "new@example.com" }
  before do
    fill_in "Name", with: new_name
    fill_in "Email", with: new_email
    fill_in "Password", with: user.password
    fill_in "Confirm Password", with: user.password
    click_button "Save changes"
  end
  it { should have_selector('div.alert.alert-success') }
end

As expected, the tests should fail until I am able to log in.

Since the Rails4 user scaffold comes with some nifty built in methods, I thought it would be easy to adapt them for my own ends:

# PATCH/PUT /users/1
# PATCH/PUT /users/1.json
def update
  respond_to do |format|
    if @user.update(user_params)
      format.html { redirect_to @user, notice: 'Profile updated.' }
      format.json { head :no_content }
    else
      format.html { render action: 'edit' }
      format.json { render json: @user.errors, status: :unprocessable_entity }
    end
  end
end

I modify the line 5 of the user controller to change the “notice” message to a “success” message:

format.html { redirect_to @user, success: "Profile updated" }

Seems reasonable, but my test for the flash message is still failing. Why won’t my page flash the user?!

Quick as a flash, I asked Dash about FlashHash.

Lo, and behold! Notice and alert are built-in flash tags, but success is nowhere to be found. How can I express the satisfaction of SUCCESS!! with a tag as mundane as notice:?

Thanks to duck-typing (if it walks like a duck and quacks like a duck, it is probably a hash.. right?) we can add our spiffy success: tag to flash, and share the appropriate level of enthusiasm with our users:

format.html { redirect_to @user, flash: { success: "Profile updated" }}

All tests are green, and today I learned that flash is

Awesome

reading, writing, rambling

Resources for New Managers

The Manager’s Path — Camille Fournier

Resilient Management — Lara Hogan

Difficult Conversations — Bruce Patton, et al

Becoming a Manager — Linda Hill

Effective Delegation of Authority — Hassan Ossman

Other Resources

So, You Want to Go to Code School!

Success With Snippets

Creating Presentations: Think of Your Audience First

Writing Technical Proposals: Exploring Multiple Options

Delivering Constructive Feedback: Stick to the Facts and Stay Curious

Snippet managers are useful for more than just code completion

Multiple Content Sources in Gatsby

Tell Gatsby about your new content

Tell Gatsby to add some node meta-data

Update createPages to generate your collections

Styling Active Links in Gatsby

Active Associations

Using Styled Components With Gatsby

Step one: ensure that gatsby knows you are using styled components

Step two: add important styles to a GlobalStyle component

Step three: Inline your GlobalStyles in your Layout component

Swoosh underline on hover (CSS)

Opaque background images

Recursive Rendering and Event Bubbling

Styled Components and Storybooks

Storybook

Styled-components

Quick start with both Storybook and styled-components

Example

My component

My stories

decorate your javascript

How does it work?

Getting started

Babel

Webpack

Linting error — or is it? 🤔

Build and run

least recently used (LRU) cache

The API/object design

The data structures

The implementaton

writing test cases in java with enums

iptables: middleware for network management

Resources

thread safety

Atomicity

Visibility

Ordering

Thread Safety Analysis Guidelines

Solutions for thread safety issues

The volatile keyword

The synchronized keyword

The final keyword

Use guava libraries

Note

effectively final

What does it mean to publish an object?

What does it mean to for an object to be immutable?

The String class is “effectively immutable”

How can I make my objects effectively immutable?

RESOURCES

annotating java

Provided by java.lang

Create your own

Annotations for Annotations

Using your CustomAnnotation

Annotating a field vs annotating a parameter

RESOURCES

Keighty's 2016 - by the numbers

browser-sync is the easiest webserver EVER

1. Install browser-sync

2. Configure the server

3. Start the web server

Service workers

How do they work?

NOTE: A single service worker can control many pages (so be careful with global variables).

The `volatile` keyword

The `synchronized` keyword

The `final` keyword

Provided by `java.lang`

How to `GET /foo`

How to `GET /foo/bar`

How to `GET /foo/42`

How to `GET /foo/42/baz`

How to `GET /foo/42/baz/123`

2. Write a Nodejs Script using `aws-sdk`