Spaghetti Programming

chef solo how to

Preface

This is a quick and basic guide I wrote that will get the opscode tool chef solo running in about 10 minutes. There are some paths and names in this guide that are specific to my use case. Obviously those would need to be changed for your environment.

1. Download and install chef

curl -L https://www.chef.io/chef/install.sh | sudo bash -s -- -v 12.3

2. Install git and clone the chef-repo

sudo apt-get install git
git clone https://github.com/opscode/chef-repo
#optionally change the remote origin url to your repo
cd chef-repo
git remote set-url origin git@github.com:MatthewRamirez/chef-repo.git

3. Create the file ~/chef-repo/solo.rb with the contents

file_cache_path "/home/matt/chef-repo"
cookbook_path "/home/matt/chef-repo/cookbooks"
json_attribs "/home/matt/chef-repo/node.json"

4. Create the file ~/chef-repo/node.json with the contents

{
  "run_list": [
    "recipe[cookbook_name::recipe_name]"
    "recipe[some_other_cookbook]"
  ]
}

5. (Optional) If you want to use knife commands like

knife cookbook create my_new_cookbook
knife cookbook site download apt

Then you will probably want to create the file ~/.chef/knife.rb.

log_level                :info
log_location             STDOUT
node_name                'ubuntu'
syntax_check_cache_path  '/home/matt/.chef/syntax_check_cache'
cookbook_path            '/home/matt/chef-repo/cookbooks'
cookbook_license         'Proprietary - All Rights Reserved'
cookbook_copyright       'Spaghetti Programming'
cookbook_email           'myemail@email.com'

7. Use this command to start chef-solo and execute the run list in node.js

sudo chef-solo -c ~/chef-repo/solo.rb

8. Edit ~/chef-repo/node.json

As you download new cookbooks from the opscode community site using knife or write your own, be sure to update the node.json with any new cookbooks you want to be included in the run list for your node.

Optional Steps to Encrypt Data Bags with Knife Solo

1. Install knife solo gems to work with encrypted data bags

You must use the gem command that is installed by the opscode chef installer in /opt to do the install.

sudo /opt/chef/embedded/bin/gem install knife-solo
sudo /opt/chef/embedded/bin/gem install knife-solo_data_bag

2. Generate encryption secret

openssl rand -base64 2048 > ~/.chef/encrypted_data_bag_secret

3. Add encrypted_data_bag_secret directive inside knife.rb

encrypted_data_bag_secret '/home/matt/.chef/encrypted_data_bag_secret'

4. Create encrypted data bags

You can create/view/edit encrypted data bags with the following commands

knife solo data bag create data_bag_name data_bag_id

knife solo data bag show data_bag_name data_bag_id

knife solo data bag edit data_bag_name data_bag_id

While the contents of this post describe working with PostgreSQL dump files specifically, these same techniques described here can be applied to searching for characters using their hex codes in any type of plain text file.

In the Postgres world, sometimes you will have to load a database dump from a database with one character encoding into a new database with a different encoding and this annoying message appears:

ERROR: invalid byte sequence for encoding "UTF8": 0xfd;

This means that there is some character somewhere in the database dump file that is not encoded for UTF8 (or whatever encoding your database is set to). In the above example the character hex code is given: 0xfd. The offending character(s) will have to be removed or converted to UTF8 encoding. You can try to do something like this using the iconv tool:

iconv -f LATIN1 -t UTF8 name_of_dump_file.dump | psql name_of_database 2>&1 | grep ERROR

This will take the contents of the dump file and try to convert the encoding of the entire file from LATIN1 to UTF8 and then pipe it into the psql command. Usually this or some other combination of from and to encoding parameters will work, but sometimes it doesn't and you have to do some more digging.

This is where the importance of the character hex value given in the error message becomes relevant. You can search the dump file for all instances of that character with grep using its hex value like so:

grep $'\xfd' name_of_dump_file.dump

The magical part of that command (at least for me) is the parameter given to grep: $'\xfd'

That parameter can be broken down into two important parts. The first part is the $ in front of the single quoted string. This is telling the shell to evaluate what is inside the quotes. The second part is the \xfd. The \x means that you are referencing a hex code, and the fd is the actual value of the character in hex. When this is all combined like above you get back the character represented by the hex code. You can even do something like this as a test to see the character printed back to you on the terminal:

echo $'\xfd'

Once you identify the location of the characters in the dump file you can decide on some way to deal with them. You could invert grep to filter them out, use sed to replace them, or something else that I don't know about.

You could also edit the file in a text editor like vim. Caution must be exercised here though because you could have an enormous file when working with database dumps and by default vim like to make swap files. You can add set noswapfile to your .vimrc or open with with -n option to disable swap files. From within vim to search by hex code you can use /\%xfd

Here again the \%x means that you are referencing a hex code and fd is the actual hex code of the character to search for.

And here is a handy ASCII cheat sheet to lookup character hex codes.

+----+-----+----+-----+----+-----+----+-----+
| Hx | Chr | Hx | Chr | Hx | Chr | Hx | Chr |
+----+-----+----+-----+----+-----+----+-----+
| 00 | NUL | 20 | SPC | 40 |  @  | 60 |  `  |
| 01 | SOH | 21 |  !  | 41 |  A  | 61 |  a  |
| 02 | STX | 22 |  "  | 42 |  B  | 62 |  b  |
| 03 | ETX | 23 |  #  | 43 |  C  | 63 |  c  |
| 04 | EOT | 24 |  $  | 44 |  D  | 64 |  d  |
| 05 | ENQ | 25 |  %  | 45 |  E  | 65 |  e  |
| 06 | ACK | 26 |  &  | 46 |  F  | 66 |  f  |
| 07 | BEL | 27 |  '  | 47 |  G  | 67 |  g  |
| 08 | BS  | 28 |  (  | 48 |  H  | 68 |  h  |
| 09 | TAB | 29 |  )  | 49 |  I  | 69 |  i  |
| 0A | LF  | 2A |  *  | 4A |  J  | 6A |  j  |
| 0B | VT  | 2B |  +  | 4B |  K  | 6B |  k  |
| 0C | FF  | 2C |  ,  | 4C |  L  | 6C |  l  |
| 0D | CR  | 2D |  -  | 4D |  M  | 6D |  m  |
| 0E | SO  | 2E |  .  | 4E |  N  | 6E |  n  |
| 0F | SI  | 2F |  /  | 4F |  O  | 6F |  o  |
| 10 | DLE | 30 |  0  | 50 |  P  | 70 |  p  |
| 11 | DC1 | 31 |  1  | 51 |  Q  | 71 |  q  |
| 12 | DC2 | 32 |  2  | 52 |  R  | 72 |  r  |
| 13 | DC3 | 33 |  3  | 53 |  S  | 73 |  s  |
| 14 | DC4 | 34 |  4  | 54 |  T  | 74 |  t  |
| 15 | NAK | 35 |  5  | 55 |  U  | 75 |  u  |
| 16 | SYN | 36 |  6  | 56 |  V  | 76 |  v  |
| 17 | ETB | 37 |  7  | 57 |  W  | 77 |  w  |
| 18 | CAN | 38 |  8  | 58 |  X  | 78 |  x  |
| 19 | EM  | 39 |  9  | 59 |  Y  | 79 |  y  |
| 1A | SUB | 3A |  :  | 5A |  Z  | 7A |  z  |
| 1B | ESC | 3B |  ;  | 5B |  [  | 7B |  {  |
| 1C | FS  | 3C |  <  | 5C |  \  | 7C |  |  |
| 1D | GS  | 3D |  =  | 5D |  ]  | 7D |  }  |
| 1E | RS  | 3E |  >  | 5E |  ^  | 7E |  ~  |
| 1F | US  | 3F |  ?  | 5F |  _  | 7F | DEL |
+----+-----+----+-----+----+-----+----+-----+

Renaming multiple files at once is something that is not done too often, but when it does come up it is a huge help to use a tool like Larry Wall's rename command. To use this simple utility, you can give rename a Perl regular expression and it will perform some very powerful file operations with just a single command. Since I don't often need to rename many files at once, I always forget how to use this utility and then end up having to look it up. Here are some examples that I find the most useful.

To change the names of all files in a directory to contain only lower case letters you simply do:

rename 'y/A-Z/a-z/' *

Doing the reverse and changing all lower case letters to capital case letters should be obvious:

rename 'y/a-z/A-Z/' *

To replace all spaces in file names with an underscore:

rename 'y/ /_/' *

Obviously be careful with this, especially when using wildcarded file names. You don't want to inadvertently rename the wrong files!

rename multiple files at once

All of this information can also be found in the man page for rename.

SSL

These are some notes on setting up https in Nginx, but they should be nearly identical for other web servers like Apache, etc.

General SSL Certificate Creation

If you want to get a real SSL certificate that doesn't sound all kinds of alarms when users visit your site then you will have to use a certificate provided by a certificate authority (CA). The CA will need a certificate signing request (CSR) to issue you a certificate and this CSR must also be signed by your key. We can use this command to generate a new key and the signing request at the same time:

openssl req -out certificate_signing_request.csr -new -newkey rsa:4096 -nodes -keyout key_file.key

Once you have the key and the CSR you can begin the ordering process with your CA. They will need to know the contents of the CSR file generated from the previous command (certificate_signing_request.csr in the above example).

If you are ok with your site throwing up warnings when someone accesses it via https, then you don't need to pay for a certificate from a CA and can instead sign the certificate yourself. This is known as a self signed certificate. This command will generate a new key and a self signed certificate:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout key_file.key -out certificate.crt

Checking Keys and Certificates

Once you generate the certificate file (certificate.crt), or obtain one from a certificate authority, you can check its contents with the following command:

openssl x509 -in certificate.crt -text -noout

To verify the contents of your key file (not so useful, but included for completeness):

openssl rsa -in key_file.key -check

Generate New Signing Request from Existing Key

If you have a single web server that hosts multiple SSL enabled sites, you can generate a new certificate signing request from an existing key:

openssl req -out new_certificate_signing_request.csr -key key_file.key -new

Configuring Nginx Virtual Host to use SSL

nginx

The certificate authority will send you the certificate as well as any intermediate certificates needed. Nginx wants the intermediate certificates in the same file as the certificate for your server. Take the contents of the SSL certificate file from your CA and save them in certificate.crt. The order of the intermediate certificates matters!

Your server must present its own certificate first and then walk up the chain of trust until it reaches the root certificate authority. This means that your certificate needs to be at the top of the file followed by the certificate that issued it. The second certificate should be followed by the certificate that issued that one and so on until the root is reached. To check the order of the certificate hierarchy when you are given a zip file of certificates you can use the openssl command to check the issuer and the subject of each certificate.

 for i in *.crt; do echo $i; openssl x509 -text -noout -in $i 2>&1| \
 egrep -i "issuer:|subject:" ; done

This will give some output like:

 AddTrustExternalCARoot.crt
    Issuer: C=SE, O=AddTrust AB, OU=AddTrust External TTP Network, CN=AddTrust External CA Root
    Subject: C=SE, O=AddTrust AB, OU=AddTrust External TTP Network, CN=AddTrust External CA Root
 COMODORSAAddTrustCA.crt
    Issuer: C=SE, O=AddTrust AB, OU=AddTrust External TTP Network, CN=AddTrust External CA Root
    Subject: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Certification Authority
 COMODORSADomainValidationSecureServerCA.crt
    Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Certification Authority
    Subject: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Domain Validation Secure Server CA
 STAR_spaghettiprogramming_com.crt
    Issuer: C=GB, ST=Greater Manchester, L=Salford, O=COMODO CA Limited, CN=COMODO RSA Domain Validation Secure Server CA
    Subject: OU=Domain Control Validated, OU=EssentialSSL Wildcard, CN=spaghettiprogramming.com

These happen to be in the correct order when the files are sorted by their name, but that may not always be the case. You can see the relationship between issuer and subject through the chain of trust and in this example simply concatenating the files in reverse order of this output will produce the correct certificate file for nginx.

If you are using a self signed certificate then there is no need to use any intermediate certs and you can simply use the self signed certificate as is.

A sample Nginx virtual host file for an SSL enabled site might look something like this:

server {
  listen   443 ;
  server_name  *.domain.com;
  ssl on;
  ssl_certificate /opt/nginx/ssl/domain.com/certificate.crt;
  ssl_certificate_key /opt/nginx/ssl/domain.com/key_file.key;
  access_log  /var/log/nginx/domain.access.log;
  root /var/www/domain/current/public;
  client_max_body_size 20M;
  passenger_enabled on;
}

Samsung 550 Chromebook
About one year ago I purchased a brand new Samsung Series 5 550 Chromebook around the time when they were first launched. While not as seductive as its now newer cousin, the Pixel, the hardware on the 550 is still really nice both on the eyes and functionally. I used it quite frequently for several months and then I noticed that the screen started to become floppy, like it had a dead spot where if you opened it up just enough, the tension on the screen to hold it in place was lost. This was rather disappointing considering it was only a few months old and being the previous flagship of the Chrombook lineup it was not exactly cheap.

So I could have RMA'ed the device through Samsung and have them repair it. I thought about that, but then decided that this process would probably take weeks, leaving me without use of the device during that period. I decided instead before sending it out to Samsung to crack the case carefully and see if I could see anything simple that might fix the issue.

I started by flipping the device over and prying off the rubber feet covering the two screws closest to the hinge with a small flat head screwdriver. If you pry carefully on the rubber feet they will stick back on just fine. Then I started to unscrew those screws. I noticed something though as I was loosening them. They were already pretty loose. So on a hunch I tightened them up and like magic the screen was now held firmly in place at all angles. Basically, I think that these screws are supposed to clamp down on the hinge mechanism somehow to keep it tight and since they were loose it allowed some slop in the hinge.

Clickable thumbs to images showing the screws that I tightened.

Remove rubber feet Screw underneath Screw closeup

The only tools used for this fix were

  • small flat head screwdriver
  • small phillips head screwdriver

And by small I mean like small enough to use on the screws in eyeglasses. Also, from memory I do not believe these screws had any thread locker on them which may have prevented them from backing out. I did not think to put any on when I was working on it, but if they come loose again I will put some on the threads. Happy fixing!