Home

Recently I got a error from my project, which reported by hoptoad, the error information like:

I18n::MissingInterpolationArgument: missing interpolation argument in "I ran %{distance}km on %{date}. It took %{time}" ({:pace=>"..."} given)

You will find I was passing the data  {:pace=>"..."} to the translation key: "I ran %{distance}km on %{date}. It took %{time}".  this translation key required 3 parameters, but I only given one. And when I look into the code, I found there is no such a mistake as I mentioned above. Then I gave up to fix this problem. Couple of weeks past, one day, when I was driving back from shanghai to hangzhou, I overheard my colleagues was discussing about this issue, they found this problem is referred to some multi-process issue. Thereafter, I did a more deeper investigation, then finally found the reason and knew how to recur it.

How to recur it?

1. memcached -vv
2. watch lsof -i :11211
3. unicorn -c config/unicorn.conf.rb -E production
4. ab -n 1000 -c 100 http://localhost:8080 (You have to run this code as soon as possible after ran unicorn command.)
5. tail -f log/production.log | grep ActionView::Template::Error

Then you would see some error messages in production.log which is the same to message wroten above.

What's the cause?

If you look carefully, when you start unicorn server, all the unicorn workers are using the same memcached connection. Then 2 http requests come, they are processed by 2 unicorn processes, then request A send many "get" requests to memcached, as well as request B. but they are using the same memcached connection at beginning, so in extreme condition, request B will get the values which is respond for the request send from request A, whereafter the tragedy happened.   However, after several errors was raised, everything goes well, no error was raised for all the time. I notice one thing: The connections to memcached of each unicorn worker was reset, each worker has its own connection, so that's why it goes correctly.

How to solve it?

After a little research online, I found the comments in unicorn config file:

after_fork do |server, worker|
  ##
  # Unicorn master loads the app then forks off workers - because of the way
  # Unix forking works, we need to make sure we aren't using any of the parent's
  # sockets, e.g. db connection

Yes, the magic code to fix this problem is:

Backlog => Sprint => Burndown => Demo => ...(Repeat)

 

What should I do after get the new feature to implement?

Generally we would get a ticket from leader (scrum master), then do it. before do it. But what should we do before start to do it:

1. We will go to understand the requirement first.
2. Make a "Accpetance test story" which must be wrote as simple and direct as possible, that client can see what you will do and confirm your understanding.
3. Seperate this big requirement into many small tickets, the best size is can be finished in one day. --- It will be easy for ScrumMaster to trace your progress and it would be more explicit about what to do in this ticket.

(Notice, don't make the tickets too small, for avoiding micro-management which is not cost-efficient.)

Have the chance to be leader.

Let all scrum members have the feeling of they are the manager of their task, but manager will be happier when they have resources to manage not only themselves, like

1. They have people to assign sub-tasks to.
2. They can make decision in their job.
3. They can make schedule for their tasks.

So it will be good method for solving this issue. Do more pair programming, for each story, one of two members can be the leader to lead other one to finish this tasks.

Ticket title or content should be none or less technical words.

Let client knows the time spent on it is worth it. You will be more comfirtable to do it.

Use Burndown chart to monitor the productivity.

This is the best way to know the health condition of the team.

Get all team members around when you do estimating.

It will be more accurate and know more about what is included in the tasks, let them estimate but not promise.

Team members should be *together*.

Together means they can Hear and See each other, also better to be seperated from other teams, that won't affect other team's work.

Give more time on quality, it worth it.

It's not about developer don't want to have a high quality work, it's about schedule made by leader didn't give them the chance.

Do retrospect after each sprint.

This will make team more health, keep a high performance and high quality work, more effective than overtime working :) Questions would be:

• Anything to improvement?
• Any problem you've found?
• Any new tech can be applied into project?

Have a break between two sprints!

This could be a good award to the team members who finish sprint in time. You just come to company do whatever that you do want to:

• Study new technology.
• Discussing anything with team members.
• Coffee
• Feet on desk

Maybe 1 day, half day, or hold a company party after work hours, or in the weekend company can offer a small trip for team members.

They are just a bunch of factory workers, but depend on their enthusiasm, did a great work by their mere knowledge, information and a pile of junk.

find /path/to/find -type f -iname *.jpg -o -iname *.css