Pitted Knowledge of the Almighty Olive: linux

Showing posts with label linux. Show all posts

Feb 19, 2018

Exposing your docker daemon API via network port (and getting it into Portainer)

These instructions will be targeted to Linux installations with systemd installed. In particular, I have used an Ubuntu-flavoured distro (ElementaryOS). I presume you have already installed docker onto your machine.

Stop the Docker daemon if is is already running
```
sudo systemctl stop docker.service
```
You can check the status of the service (including if it is even installed)
```
sudo systemctl status docker.service
```

Open the service configuration file

sudo nano /lib/systemd/system/docker.service

Find the line with 'ExecStart' and modify it as follows (saving it once complete):
```
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H fd://
```
Reload all of the daemons:
```
sudo systemctl daemon-reload
```
Start the service
```
sudo systemctl start docker.service
```
Open up your portainer installation, navigate to the 'Endpoints' menu item and then enter in the IP and port for your target computer.

Jul 24, 2017

PHP XDebug tips and tricks

A quick step-by-step walkthrough on how to get XDebug working properly and interfacing with your IDE of choice.

Install XDebug on the PHP server (or docker instance). This will enable XDebug on your PHP code.
```
apt-get install --no-install=recommends php5-xdebug
```

Configure /etc/php5/mods-available/xdebug.ini on the server to enable XDebug (check the documentation for more details about the settings).


zend_extension=xdebug.so
xdebug.remote_enable=1
xdebug.remote_autostart=1
xdebug.remote_connect_back=1
xdebug.remote_host=10.0.0.1
xdebug.max_nesting_level=500

Ensure that the port 9000 (you can change this in the above config file) is open on the server (configure your firewall). If PHP is running as a docker service, you may need to expose/map the port and then configure your firewall to allow connections.
Download or install a client program to pass the XDebug IDEKEY to the server while browsing
Configure your IDE to listen on port 9000 of the server for XDebug messages

Note: You should use the browser on the same machine you have your IDE set-up, as PHP XDebug will send messages to that and only that client (depending on the configuration options).

Jul 4, 2014

Recover LVM Boot partition within a live CD (installing GRUB)

This allows you to recover the Master Boot record when a RAID 5 array fails.

Activate LVM partitions
sudo mdadm --assemble --scan
sudo vgchange -a y [name of volume group]
Mount all volumes to a single mount-point
sudo mount /dev/mapper/[volume group]/[root volume] /mnt/
sudo mount /dev/mapper/[volume group]/[usr volume] /mnt/usr/
sudo mount /dev/mapper/[volume group]/[home volume] /mnt/home/
... etc
Mount and bind directories for grub to detect the underlying hardware
sudo mount --bind /dev /mnt/dev
sudo mount --bind /dev/pts /mnt/dev/pts
sudo mount --bind /proc /mnt/proc
sudo mount --bind /sys /mnt/sys
Jump into the new mounted system
sudo chroot /mnt
Install GRUB
grub-install /dev/sda
grub-install --recheck /dev/sda
update-grub
Exit the environment and restart
exit
sudo umount /mnt/dev/pts
sudo umount /mnt/*
sudo shutdown -r now

Jul 3, 2014

Recovering a RAID 5 array

Recently had a RAID 5 array fail on me. These are the steps I took to recover the data.

NOTE! The order of the disks in /dev/sdX follows the numbering of the ports on the motherboard. So if a disk is plugged into port 0, it will show up as /dev/sda. Keep this in mind as if you remove a failed disk, it might mess up the references in the array!.

Boot up from a Live CD
Install the software RAID management software
sudo apt-get install mdadm
Make sure RAID and LVM are unmounted
sudo vgchange -a n [name of your volume group] sudo mdadm -S /dev/md0
Copy the failed disk to the new disk (in case the disk was the boot disk, you need to copy that flag across so that your system boots)
sudo sfdisk -d /dev/sdx | sudo sfdisk /dev/sdy
Check that the disks are the same
sudo fdisk -l
Mount the RAID array, and remove the failed disk
sudo mdadm --assemble --scan sudo mdadm --manage --remove /dev/sdx
Add the new disk
sudo mdadm --manage --add /dev/sdy
Watch the progress as mdadm rebuilds your RAID array
sudo cat /proc/mdstat

Nov 30, 2013

Debian and XFCE: Print Screen Key

Print Screen Key

To enable the print screen key, just head on to Application Menu > Settings > Keyboard and click on the Application Shortcuts tab.

Add a new command and in the text box enter in the following:

xfce4-screenshooter -f

Hit Ok, and the next dialogue will require you to enter in the key (or key combination) that will trigger this command (in this case, we will press the 'Prt Scr' key).

References:

https://wiki.archlinux.org/index.php/Xfce

Nov 4, 2013

Debian: Installing gitolite

Tried setting up gitolite in Debian following my earlier guide with CentOS, but it didn't work out as well because there were a few steps missing (Debian doesn't create the gitolite user automatically). These are the modified steps:

Install gitolite:
sudo apt-get install git git-core python-setuptools gitolite
Add the gitolite user (this is the user that will host the git repositories and control access):
sudo useradd gitolite
As the user who will be administrating the set-up (i.e. someone OTHER than the gitolite user), create a set of ssh keys and move them to where the gitolite user can access them:
ssh-keygencp ~/.ssh/id_rsa.pub /tmp/admin.pub
Switch to the gitolite user:
sudo su - gitolite
Run the set-up command:
gl-setup /tmp/admin.pub
As the user who will be administrating the set-up, clone the repository
git clone gitolite@[ip address]:gitolite-admin.git

You can now add in other keys and set up the configuration of gitolite.

References:

Nov 3, 2013

Debian and XFCE Dual Monitors

Open up a terminal and type in xrandr. This will output the configuration of the currently connected monitors. Identify the names of your monitors (such as HDMI2, LCD1, VGA1, CRT2 or whatever). In this example we have two monitors, HDMI1 and HDMI2.

Now type in the desired layout:

xrandr --output HDMI2 --left-of HDMI1

The monitors should now be showing your desired screen laytout. If not, play around until you have it.

To make this layout permanent, click on the Start menu > Settings > Session and Startup and then click on the Application Autostart tab. Add a new autostart item and then insert the xrandr command into the Command text field. Now if you restart your settings will be automatically loaded!

References

http://www.linuxquestions.org/questions/showthread.php?p=4658799

Nov 1, 2013

Debian: Sound issue

I had a bit of an issue with ALSA and Debian in regards to my sound device not being detected properly. This meant I had to dig around a bit to get it up and running.

Note that with this method may not work nicely with multiple sound cards. Your mileage may vary....

Problem:

Running the command alsamixer in a terminal returned this error message:

cannot load mixer controls: Invalid argument

Trying to restart the ALSA server got me this:

root@localhost:~# /etc/init.d/alsa-utils stop
[....] Shutting down ALSA...warning: 'alsactl store' failed with error message 'alsactl: get_control:250: Cannot read control info '2,0,0,Front Playback Volume,0': In[FAIL argument'...failed.

So there was something wrong with the system configuration or the control.

For reference, this is what my system has configured:

root@localhost:~# head -n 1 /proc/asound/card0/codec*
==> /proc/asound/card0/codec#2 <==
Codec: Realtek ALC887-VD

==> /proc/asound/card0/codec#3 <==
Codec: Intel PantherPoint HDMI

root@caesar:~# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: ALC887-VD Analog [ALC887-VD Analog]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 1: ALC887-VD Digital [ALC887-VD Digital]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 3: HDMI 0 [HDMI 0]
Subdevices: 1/1
Subdevice #0: subdevice #0

Explanation of these commands (skip if you want to get to the solution)

The first command reads out the first line (head -n1) of each of the files that match the search pattern (/proc/asound/card0/codec*). This matches two files (codec#2 and codec#3). The /proc/ filesystem is where Linux stores all the files necessary for running the operating system, including detected hardware devices. The first file tells us that the system recognizes the motherboard sound card, and the second files recognizes the HDMI output.

The second command confirms our suspicions, and also let's us know that ALSA detects these devices too (the Realtek and the HDMI devices are found and configured).

Now we know that the problem lies in the ALSA configuration, because everything is detected at this stage.

Solution:

Run the following command and look for your audio device:

lspci -v
The above step is important because you need to find the line 'Kernel driver in use' associated with your Audio controller (which is outputted by the above line). In my case the kernel was using snd_hda_intel.
Now we need to check to ensure the driver is installed. A simple way is to start off writing (DO NOT PRESS ENTER YET!) modprobe snd and then tab complete it (press the <TAB> key twice). This should bring up a list of sound drivers your system knows about. If your driver is not listed you should try to find and install it.
Edit /etc/modprobe.d/alsa_base.conf and add the following line (you may need to adjust this to suit your system):
options snd-hda-intel model=generic
Now we need to force ALSA to load the new configuration:
alsa force-reload
If you can hear sound after running the following command, congratulations!
aplay /usr/share/sounds/alsa/Front_Center.wav

References:

Aug 25, 2013

Linux Mint Multiseat with keyboards & mice(Xephyr)

I must state this important fact first:

Xephyr on Ubuntu/Linux Mint does not come compiled with evdev support.

This is important because Linux uses evdev to configure most input devices like keyboards and mice. You will see lots of documentation on configuring inputs with evdev, but none of those methods will work unless you compile Xephyr from scratch and enable evdev yourself.

I downloaded and installed the following executable to make my life easier. I repeat, this method will not work unless you have a modified version of Xephyr with evdev support!

Correct drivers - make sure they are installed!

I used an older Nvidia card so the following commands got me up and running:

sudo apt-get -y xserver-xorg-video-nouveau
#
# Upgrade our system
sudo apt-get -y install ubuntu-drivers-common
sudo apt-get -y install nvidia-current nvidia-settings

If you are unsure what graphics card you have then run the following (the second line is my output; yours will probably be different):

$ lspci | grep VGA
01:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)

Input devices

Determine which devices are which by running the following command (unplug devices to help to narrow down your options). The paths you see are going to help in writing up our configuration file.

$ ls -l /dev/input/by-path/
total 0
lrwxrwxrwx 1 root root 9 May 27 18:39 pci-0000:00:1d.0-usb-0:1:1.0-event-kbd -> ../event3
lrwxrwxrwx 1 root root 9 May 27 18:39 pci-0000:00:1d.0-usb-0:2:1.0-event-mouse -> ../event4
lrwxrwxrwx 1 root root 9 May 27 18:39 pci-0000:00:1d.0-usb-0:2:1.0-mouse -> ../mouse0
lrwxrwxrwx 1 root root 9 May 27 18:39 platform-i8042-serio-0-event-kbd -> ../event2
lrwxrwxrwx 1 root root 9 May 27 18:39 platform-i8042-serio-1-event-mouse -> ../event5
lrwxrwxrwx 1 root root 9 May 27 18:39 platform-i8042-serio-1-mouse -> ../mouse1

Custom Xephyr script

The custom script (save it to /usr/sbin/Xephyr.sh) will act like the glue in our multi-seat environment. It will attach input devices to our monitors and some other stuff.

#!/bin/bash
# 20060905 - josean - added get_event() function to obtain eventNN from a physical address
# Original version:
# http://en.wikibooks.org/wiki/Multiterminal_with_Xephyr
# http://www.c3sl.ufpr.br/multiterminal/howtos/Xephyr.sh
trap "" usr1
XEPHYR=/usr/local/sbin/Xephyr
get_event()
{
evento=`grep -A5 $1 /proc/bus/input/devices | grep 'H: Handlers=' | grep --only-matching -e 'event[0-9]*'`
}
args=()
while [ ! -z "$1" ]; do
if [[ "$1" == "-xauthority" ]]; then
shift
if [ ! -z "$1" ]; then
export XAUTHORITY="$1"
fi
elif [[ "$1" == "-display" ]]; then
shift
if [ ! -z "$1" ]; then
export DISPLAY="$1"
fi
elif [[ "$1" == "-kbdphys" ]]; then
shift
if [ ! -z "$1" ]; then
get_event $1
args=("${args[@]}" "-keybd")
args=("${args[@]}" "evdev,,device=/dev/input/$evento,xkbrules=evdev,xkbmodel=evdev,xkblayout=us")
fi
elif [[ "$1" == "-mousephys" ]]; then
shift
if [ ! -z "$1" ]; then
get_event $1
args=("${args[@]}" "-mouse")
args=("${args[@]}" "evdev,5,device=/dev/input/$evento")
fi
else
if ! expr match $1 'vt[0-9][0-9]*' >/dev/null; then
args=("${args[@]}" "$1")
fi
fi
shift
done
echo $XEPHYR "${args[@]}"
exec $XEPHYR "${args[@]}"

Xorg.conf settings

Edit /etc/X11/xorg.conf with something similar to the following. Please change the values to your whatever matches your system!!

############## SETTINGS#############
Section "ServerFlags" Option "DontZap" "true" Option "DontVTSwitch" "true" Option "DontZoom" "true" Option "AllowMouseOpenFail" "true" Option "AllowEmptyInput" "true" Option "AutoAddDevices" "false" Option "AutoEnableDevices" "false" Option "Xinerama" "false" Option "NoPM" "true" Option "DPM" "false" Option "BlankTime" "0" Option "StandbyTime" "0" Option "SuspendTime" "0" Option "OffTime" "0"EndSection
############## INPUTS#############
Section "InputDevice" Identifier "Keyboard1" Driver    "evdev" Option    "Device" "/dev/input/event2" Option    "Floating" "true" Option    "XkbRules" "evdev" Option    "XkbModel" "evdev" Option    "XkbLayout" "us"EndSection
Section "InputDevice" Identifier "Mouse1" Driver    "evdev" Option    "Device" "/dev/input/event5" Option    "Floating" "true" Option    "GrabDevice" "on" Option    "Protocol" "auto" Option    "Emulate3Buttons" "no" Option    "ZAxisMapping" "4 5"EndSection
Section "InputDevice" Identifier "Keyboard0" Driver    "evdev" Option    "Device" "/dev/input/event3" Option    "Floating" "true" Option    "XkbRules" "evdev" Option    "XkbModel" "evdev" Option    "XkbLayout" "us"EndSection
Section "InputDevice" Identifier "Mouse0" Driver    "evdev" Option    "Device" "/dev/input/event4" Option    "Floating" "true" Option    "GrabDevice" "on" Option    "Protocol" "auto" Option    "Emulate3Buttons" "no" Option    "ZAxisMapping" "4 5"EndSection
########### SEAT 1##########
Section "Device" Identifier "Device1" Driver    "nvidia" Vendorname "NVIDIA Corporation" BoardName "GeForce 210" Option    "DPMS" "false" Option    "UseDisplayDevice" "CRT" Option    "ProbeAllGpus" "false" Option    "NoLogo" "true" Option    "RenderAccel" "true" Screen    1EndSection
Section "Monitor" Identifier "Monitor1" VendorName "Toshiba" ModelName "Toshiba Matsushita Display Technology Co., Ltd LCD-MONITOR" Option    "DPMS" "false"EndSection
Section "Screen" Identifier "Screen1" Device    "Device1" Monitor    "Monitor1" DefaultDepth 24 Subsection "Display"    Depth 24    Modes "nvidia-auto-select" EndSubsection Option    "DPMS" "false" Option    "UseDisplayDevice" "CRT" Option    "ProbeAllGpus" "false"EndSection
############# SEAT 0############
Section "Device" Identifier "Device0" Driver    "nvidia" VendorName "NVIDIA Corporation" BoardName "GeForce 210" Option    "DPMS" "false" Option    "UseDisplayDevice" "DFP" Option    "ProbeAllGpus" "false" Option    "NoLogo" "true" Option    "RenderAccel" "true" Screen    0EndSection
Section "Monitor" Identifier "Monitor0" VendorName "Toshiba" ModelName "Toshiba Matsushita Display Technology Co., Ltd LCD-MONITOR" Option    "DPMS" "false"EndSection
Section "Screen" Identifier "Screen0" Device    "Device0" Monitor    "Monitor0" DefaultDepth 24 SubSection "Display"    Depth 24    Modes "nvidia-auto-select" EndSubsection Option    "DPMS" "false" Option    "UseDisplayDevice" "DFP" Option    "ProbeAllGpus" "false"EndSection
############## SERVERS#############
Section "ServerLayout" Identifier "multix" Screen 0 "Screen0" 0 0 Screen 1 "Screen1" 0 0EndSection

MDM configuration

The MDM is what executes everything (if it has been configured properly). Edit /etc/mdm/mdm.conf and change the Server Section to the following:

## Also note, that if you redefine a [server-foo] section, then MDM will# use the definition in this file, not the MDM System Defaults configuration# file. It is currently not possible to disable a [server-foo] section# defined in the MDM System Defaults configuration file.#
[server-Xephyr0]name=Xephyr0command=/usr/bin/X -ac -br -layout multix -audit 4 -dpmshandled=falseflexible=false
[server-Xephyr1]name=Xephyr1command=/usr/sbin/Xephyr.sh -display :0.0 -xauthority /var/lib/mdm/:0.Xauth -fullscreen -kbdphys usb-0000:00:1d.0-1/input0 -mousephys usb-0000:00:1d.0-2/input0 -verbosity 100 -audit 4 -screen 0 -dpms -retrohandled=trueflexible=false
[server-Xephyr2]name=Xephyr2command=/usr/sbin/Xephyr.sh -display :0.1 -xauthority /var/lib/mdm/:0.Xauth -fullscreen -kbdphys isa0060/serio0/input0 -mousephys isa0060/serio1/input0 -verbosity 100 -audit 4 -screen 1 -dpms -retrohandled=trueflexible=false

Resources

Aug 23, 2013

Coursera Notes: Stanford 'Start-up Engineering' (Lectures 5-8)

These are just some of my notes from Coursera's 'Start-up Engineering' course, taught by Balaji Srinivasan from Stanford.

This is a continuation of my existing series of notes.

Market Research, Wire-framing and Design

Idea $\ne$ Mock-up
Mock-up $\ne$ Prototype
Prototype $\ne$ Program
Program $\ne$ Product
Product $\ne$ Business
Business $\ne$ Profit
Execution! It is not the idea, but the execution that matters. Sales rather than technology is what builds a business.
Market! Market will draw a product from a team, whether or not it is quality or the team is good.
An idea exists within a maze. A simple sentence is not enough to describe an idea; an idea is defined by the regulations, markets, and competition.
Execution mindset. This is essentially writing a to-do list and regularly checking off items. Rinse and repeat.
Market Research:

News coverage and research papers. Google Books, SEC filings and Wikipedia.
Back-of-envelope estimate of market size. Look for relevant statistics.
Validate. Google keyword planner and Facebook advertiser tools help determine if there is actually a market need.
Do a basic launch page with basic SEO. Use wireframes.
Ad-word to discover the market. The launch page will then gauge market interest.

MVP or Minimum Viable Product.
Remember, a start-up aims to be very ambitious and scale rapidly.
Two features of successful start-ups:

Exhibit economies of scale. Cost of production per unit decreases as more units are built (but revenue stays the same). We can then determine a break-even point and therefore the minimal capital required.
Attack/Pursue large markets. Different pricing will attract different markets, but low price points require automation and industrial efficiency to make profits (because customer service is expensive). It may be better to charge higher initially to counter risks. Market sizing calculations should be done early and often.

Once a market and broad perspective has been set, versions and features need to be prioritized. Remember, it is execution and sales that matter!
Rough guide to prioritizing versions and features:

How much are they willing to pay for certain features or versions?
Which features are required in each version? What features make sense to bundle together?
Estimate the time and cost to build each feature. Is it feasible to implement the feature now, or wait for more funding?
Find the most popular features.
Calculate the market size for each feature.

Wireframing tools: omnigraffle, lucid chart, jet strap and popapp.
Copy-writing:

Home-page message must allow a customer to immediately figure out what the product is. This is a priority if this is going to be a major source of potential customers.
Work backwards from the press release (write the release then build the product). This allows you to figure out which features are making the news and which are not.
Find your competitors and explain why they are terrible options. Use this insight when explaining the benefits of your product.
Simple, factual and concise statements.
Call to action. Allow the customer to do something once they visit your website.

Vector graphics are better to work with.
In design remember Alignment, Repetition, Contrast and Proximity.
Start with a font heavy design (it is easier to do and images can always come later)

Mobile

Assumption behind the mobile phenomenon is that everything is going to be on the internet. The internet is going from a novelty to a utility.
Build for HTML5 and then move to native apps. HTML5 ensure your application works on all devices (and Android will soon utilise HTML5 and Javascript instead of native applications).
Internet of Things is the idea that every device will have it's own IP address. This offers a huge potential market.
Quantified self is the measuring of human beings and our actions. This is the collection of metrics that may revolutionize diagnosis and medicine.
One way to build mobile-aware applications is user-agent sniffing. This approach has the problems that a client can fake their own user-agent, and that the user-agent is inherently unreliable.
CSS media queries and Responsive web design allows the application of conditional styles depending on screen size. This is much more reliable, but does not have ubiquitous support (yet).
Some constraints with mobile include:

Unreliable networks (the fallacies of distributed computing)
Debugging requires logging (and bug reporting)
Minimization of user input (difficult problem to solve; how to collect everything you need without overwhelming the user)
Minimize the time to result (if you take too long the user will go elsewhere)

HTML / CSS / Javascript

HTML is the skeleton of a web application. It provides the structure of a page and the semantics. It is a set of finite elements with attributes.
CSS is the look and layout of a web application. It edits the element and attributes for styling and formatting.
Javascript is the dynamics and behavior of a web application. It allows you to provide client-side validation, pulling in content, playing games and much more.
Some useful tools include jsfiddle.net and Chrome Developer Tools.

Deployment, DNS and Custom Domains

Your code production environments should be along the lines of Development -> Staging -> Production
Separating environments bring the following benefits:

Testing of features before they reach the customer
Roll back of code in case of major bugs
Restore code or data in case of catastrophic crashes of the server
Incorporate contributions from multiple engineers
Perform AB testing of features

DNS (Domain Name System) converts IP address into human readable hostnames. The system first looks locally in a program, then the OS, then the ISP and then finally a trusted internet DNS server.

Aug 16, 2013

Coursera Notes: Stanford 'Start-up Engineering' (Lectures 1-4)

These are just some of my notes from Coursera's 'Start-up Engineering' course, taught by Balaji Srinivasan from Stanford.

Lecture 1:

A start-up is typically a company designed to grow rapidly and scale to global markets. The idea is to take ownership of the market before competitors move in.

The modern Start-up company is generally focused around the internet and other emerging technologies, but history is filled with start-up stories. The automobile, aviation, oil, and pharmaceutical industries all began with similar start-up journeys. Some of the biggest businesses we know had very humble beginnings.

The modern start-up industry began in 1989-1992. This was due to a combination of factors such as the widespread adoption of the internet, the fall of the USSR and the repeal of the NSF AUP in the USA.

The USSR (and other communist regimes) heavily regulated the use of technology, and you could be jailed for any unauthorized use. The fall of the USSR allowed the adoption of technology by the mainstream population, vastly increasing the amount of people interacting on-line. It also forced India to introduce policies that deregulated technology use and made China focus on market reforms. These factors helped to create the global free market.

The National Science Federation (NSF) in the USA had banned e-commerce because of fears of malware, spam and pornography. Eventually the US congress repealed the Accepted Use Policy (AUP), and this allowed people to start trading goods and services on-line.

The features of a modern start-up company include:

Operational Scalability: This refers to the ability to conduct transactions from anywhere in the world without requiring a physical presence. This means you can rapidly expand into global markets without increasing operational overhead.

Market size: The internet means a company now has access to a customer from anywhere in the world. This exponentially increases your market size (as long as border and geographical restrictions are solved).

Generality: Software is a flexible and malleable tool with almost limitless potential. Software skills are also portable.

Low capital barriers: Hardware costs are relative cheap, so sophisticated equipment can be bought with little capital overhead. Developers can also create their own software tools to suit the job at hand.

Low regulation barriers: It is currently very hard to regulate the internet, but this should not be taken for granted. The firewall in China and the USA's NSA spying program are examples of attempts by governments to control, regulate and restrict the internet.

Open source: The internet is built on open source technologies such as DNS, HTTP, HTML, IP, DHCP and other protocols and specifications. The free exchange of ideas and common technologies means the rapid emergence of new and useful tools for the entrepreneur.

The long trail: The global scale of the potential market means that a start-up can target extremely specific customers and market niches like never before.

Failure tolerance: Penalty for failure is lower than other industries (such as automobiles and aviation).

Able to build a hybrid business: Can supply an API to interact with third-parties or the physical world. Automation through device drivers and actuators.

The current trend for start-ups is towards mobility and decentralization (or at least reducing the penalty for location and nationality).
Start-up engineering is focused on shipping a workable product. Iterative development is key; ship an initial product with reduced functionality to bring in some early funding to fuel further improvements in the next version.
Primary task of a start-up engineer is the integration of diverse technologies. They need to keep up with the latest developments, evaluate the usefulness of technology and quickly snap together the pieces.
Engineers need versatility, especially with Design, Marketing and Sales.
Mobile HTML5 and JS/JSON is the future of web applications. They allow for responsive mobile design (with a desktop UI as an aftereffect), which allows the use of the application on as many devices as possible.

Lecture 3 & 4:

Virtual Machines allows us to take a single physical computer and make it seem like multiple independent computers. Virtualisation significantly reduces the infrastructure overhead.

Linux has server-side license loophole. This allows a developer to modify open source code without distributing those changes to the public. This means you can modify code to create a service without releasing those code changes, as long as you are not distributing the changes for profit.

The Cloud Computer is a computer whose precise physical location is immaterial to the application. There are three approaches:

IAAS: (Infrastructure As A Service) Direct access to hardware

PAAS: (Platform As A Service) API access to the hardware.

SAAS: (Software As A Service) API and GUI to the application, but no control over the hardware.

$PATH is the order of directories that Linux will use to search for a command. The first matching command found is assumed to be the desired command.
'which' is a useful command to determine which command Linux will find first. This can help when you have multiple versions of a command installed
bash is a command-line shell implementation
A shell script begins with a sha-bang (#!) followed by the path to the command that the shell will use to interpret/execute the script.
ssh allows you to securely connect to a remote machine and run commands
scp allows you to connect and copy files to a remote machine
You can configure SSH so that you don't have to write out the connection details every time. Add the following to the file ~/.ssh/config and invoke with ssh <name>

Host <name>
HostName <hostname>
User <username>
Identityfile <the path to the file>

STDIN is the input stream, STDOUT is the output stream, and STDERROR is the error stream
Some useful linux commands:

cd - change directory
alias - set a command alias to save typing
rm - remove a file
mv - move a file
mkdir - create a directory
pwd - print the current working directory
env - list all environment variables
ls - list files in current directory
ln - create symbolic links
rsync - synchronise a local file with a remote file
wget - download a file (unlike rsync this is only for publicly available files)
curl - Only for single URLs, and can support more protocols than wget
ping - test network availability
less - used to view large files by paging it. CTRL+N down, CTRL+P up, Q quit
cat - File viewer, but does not have pagination features of less
head - view first few lines of a file
tail - view last few lines of a file
cut - Pull out columns from a file
paste - paste data into columns
nl - print our the line number
sort - sort lines in a file
uniq - determine unique elements in a file
wc - line, word and character count
split - split large files
man - single page manual files for commands
info - for some applications this will provide more detail than what man provides
uname - lists system information
hostname - name of machine
whoami - name of current user
ps - list current running processes
kill - kill a process
top - list processes based on criteria
sudo - act as root user for one or more commands
su - become root user
tar - archival utility
gzip - compression utility
find - non-indexed file search
locate - indexed file search. Requires updatedb command to be operational
df - determine disk space
du - determine file's disk usage
grep is a text and file parser that uses regular expressions. Very powerful.
sed is a string substitution command. Used to do a find and replace.
awk is a useful scripting language for tab-delimited text.

A list of useful bash shortcuts:

CTRL+K : Kill everything from cursor up
CTRL+C : Abort command
CTRL+L : Clear the screen
CTRL+D : Exit the command prompt

Backticks ` allows you to use results from commands as part of a new command
Ampersand & allows you to run a command in the background
xargs allows you to build command line arguments, and can spawn parallel processes.
tee allows you output to both a file and the display
time is useful for bench-marking commands
screen is a manager for remote tabs. This allows you to save a context that allows you to resume your work if you lose connection temporarily.

Dec 1, 2012

Migrating and transfering gitolite to a new server

I am going to assume you have already installed gitolite on two servers (if you haven't, check out my guide on installing gitolite). This post will outline how you will move your repositories from one server to your new one.

The server side

You can edit some of the configuration files stored in the home directory of the gitolite user. You only have to do this step if you have an unusual set-up; it should work fine if you did a default install.
Copy the contents of your repositories folder to the new server (except gitolite-admin; we will do this at a later stage). Use either the commands cp or scp. For instance:
scp -r repo.git/ root@192.168.0.1:/var/lib/gitolite/repositories/
Change the owner and group of the copied repositories:
chown -R gitolite:gitolite repo.git/
Clone the gitolite configuration repository on your administration machine:
git clone gitolite@192.168.0.1:gitolite-admin.git
Add the keys and configuration files from your old repository and place them into your new repository. Do a commit and then a push:
git add .
git commit -as
git push
The server is now set up and ready to use! Optional: The guide quoted that you may need to run gl-setup again once the repositories have been copied across, but I didn't need to. You may want to do that step....

The client side

To point your git repository to the new server (so you don't have to reconfigure your IDE or scripts) just run the following commands:

git remote rename origin old
git remote add origin git@192.168.0.1:repo.git
git remote -v
git remote rm old

Nov 30, 2012

Installing gitolite in CentOS 6

Gitolite is an management service that sits on top of git. It helps restrict users to certain projects (and what they can do on those projects). In this post we will install gitolite in a CentOS 6 environment.

First we need to enable the EPEL repository. You could download and install gitolite directly, but I prefer to manage everything through the package manager for auditing purposes.
wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-7.noarch.rpm
rpm -ivh ./epel-release-6-7.noarch.rpm
Install gitolite (it will most likely install a variety of dependencies):
yum install gitlolite
If this is a brand new gitolite installation you will need to create a public SSH key on the account you will be using to administer your gitolite installation. The creation of these keys are outside the scope of this documentation. Once the key pair has been created, copy the public version to a common place where gitolite can access it (like temp). Use the command cp or scp to acheive this.
Rename your copied public key with some sort of identifier. Gitolite uses the name of your keys to determine access.
Log in as the gitolite user:
su - gitolite
Initialize your gitolite service with the key:
gl-setup -q /tmp/user.pub
Now you can use gitolite!
ssh gitolite@192.168.0.1 info
git clone gitolite@192.168.0.1:gitolite-admin.git

Nov 28, 2012

Bridge Networking in CentOS 6.3

Bridge networking is a useful technique to allow Virtual Guests to access your networking hardware. This guide was written in mind for CentOS 6.3 but should be applicable to other Linux versions (with modifications).

Copy the file /etc/sysconfig/networking-scripts/ifcfg-eth0 as br0
cp /etc/sysconfig/networking-scripts/ifcfg-eth0 /etc/sysconfig/networking-scripts/ifcfg-br0
Edit the file /etc/sysconfig/networking-scripts/ifcfg-eth0 and add the line:
BRIDGE=br0
You can also delete the lines:
BOOTPROTO
IPADDR
GATEWAY
DNS1
DNS2
Edit the file /etc/sysconfig/networking-scripts/ifcfg-br0 and edit the lines:
DEVICE=br0
TYPE=Bridge
You can also delete the lines:
HWADDR
UUID
Restart your network:
service network restart

References

Howtoforge.com guide to Virtualisation with KVM
Cyberciti post on KVM Bridging
The isborken blog on setting up a headless KVM

Nov 23, 2012

Setting up a CentOS 6 server: Services Hardening

This is a follow on post from my guide to installing CentOS 6.2 (or you can read my updated 6.3 version). You can see my other posts such as auditing your software installs, hardening your accounts, network hardening and clearing out orphaned packages.

This guide outlines how to cut down on unnecessary services so that you have a lean and mean machine.

List all the services running on your machine with the following command:
chkconfig --list | grep :on
Go through the list and select packages to disable or remove. For instance:
chkconfig mdmonitor off
chkconfig smartd off
chkconfig messagebus off
chkconfig haldaemon off
chkconfig cups off
chkconfig atd off
chkconfig kdump off
If you do not know what a service is or does, just run:
rpm -qf /etc/init.d/<service_name>

Then run:
rpm -qi <rpm>

References

Red Hat Linux 5 Hardening Tips - National Security Agency
Guide to the secure configuration of Red Hat Linux 5 - National Security Agency
CentOS wiki on hardening the OS
This guide from sysadminwiki

Nov 20, 2012

Clearing orphaned and unused packages from CentOS 6.3

This is a follow on post from my guide to installing CentOS 6.2 (or you can read my updated 6.3 version) and auditing your software installs. We will go through some of the steps required to secure your server and get it ready for production use.

As always, I suggest you take this time to tighten up your machine first; run updates, turn off services, install software and harden your machine. You should also consider setting up your SSH settings.

To check which packages are left on your system just run the following command:

package-cleanup --leaves --exclude-bin

(NOTE: The --exclude-bin option means that packages with bin files are not included; to see packages with bin files just delete the option)

If you are happy with the list produced, run the modified version to delete all the files:

package-cleanup --quiet --leaves --exclude-bin | xargs yum remove -y

Nov 15, 2012

Setting up a CentOS 6.3 Virtual Host

This guide outlines how I set-up my virtual host using CentOS 6.3 (a free release version of Red Hat Linux with all the branding removed). It's partially based on my previous installation guide. I also used my guide to create a bootable USB installation disk, but this guide should work equally well with the standard DVD install.

Installation

Boot up your installation media (you may need to edit your BIOS to do so)
Select 'Install or upgrade an existing system' from the menu
Select your language and keyboard layout.
Click next at the splash screen.
Choose the 'Basic Storage Device' option
Select the 'Fresh Installation' option
Enter in the host-name of your new server (for best results you should append your domain name to the end so it works seamlessly with SSL certificates) i.e. testserver.example.com

If you want to configure a static IP address click on the 'Configure Network' button, select the your network card (probably eth0) and enter away.

If you are going to use DHCP, or just don't know, just hit 'Next'
Select the correct timezone for you (just click a location on the map and it should select the closest one to you).
Enter in an appropriate root password. Make as long and complex as possible (long sentences with mixed character types are easier to remember than jibberish strings; for instance 'My office is situated in 1234 fake street, Fakeville!')
In this example we are going to go for a custom partition layout, so select 'Create Custom Layout'. If you are fine with defaults, just skip to part .
Delete all existing partitions and do the following:

A /boot partition of about 100MB. Use the ext4 format

Create a LVM Physical Volume that fills up the rest of the hard-drive

Create a LVM Volume Group with a Physical Extent of 4MB.

Create LVM Logical Volumes on the Volume group as follows (make sure you leave some free space for your Virtual machines!!):

Swap space that is at least equal to how much RAM is in your server
/tmp/ should be as big as the largest file you will be manipulating (for instance, if you are copying a DVD you will need at least 4GB)
/var/log and /var/log/audit are separated so that if your log system goes haywire it does not kill the space for other applications. Dedicate a couple of gigabytes to each.
/home/ and /usr/ should be a few gigabytes each. /usr/ just holds your applications and should remain pretty static, while /home/ is where you will store your personal files.
/var/ and /var/www/ will contain the majority of space on your system. MySQL stores your database files in /var/lib/mysql/, while Apache runs from /var/www/. Dedicate adequate space to each folder.
Your root folder (/) will only need a few GB of space. It will mainly hold configuration files.

The system will take some time to format your hard-drive. Once it is complete it will ask you to install the boot-loader. While the defaults are suitable, for extra security you should consider password protecting your boot-loader.
We can now select our packages. You can customize the system to suit your needs, but for the basics just select 'Basic Server' from the menu and the 'Customize now' from the radio buttons. Hit 'Next'.
Do the following edits:

Remove the 'Java Platform' and 'Directory Client' meta-packages
Add all of the Virtualization meta-packages (including client, platform and tools)
Because the virt-manager tool requires a GUI, you may need to install the 'X Windows System' and the 'KDE Desktop'
From the base system, I removed packages such as hunspell and word (as well as hardware tools like RAID that I was not using)

Reboot your system!

House cleaning

I suggest you take this time to tighten up your machine; run updates, turn off services, install software and harden your machine. You should also consider setting up your SSH settings.

Have a look at some of my guide to Software package integrity checks (aide).

Creating our first guest

We are going to use LVM based guests, so if you haven't left any space on your LVM partition I suggest you use these guides to free up some space. If you have partitions you don't think you need anymore, just delete it.

You may also want to ensure that KVM is installed so that you get the benefit from it's kernel and hardware virtualisation.

yum install kvm qemu-kvm qemu-kvm-tools

Now you just need to create a logical partition in the volume to store your VM by running the following command (assuming your volume is call VolGroup):

lvcreate -L20G -n vm1 VolGroup

To install to this new partition, just run the following command:

virt-install --connect qemu:///system -n vm1 -r 512 --vcpus=2 --disk path=/dev/VolGroup/lv_vm1 -c /path/to/installation.iso --graphics vnc --noautoconsole --os-type linux --os-variant rhel6

Note the following parameters:

-r specifies the RAM size
--vcpus specifies the virtual CPU's to use
--os-type helps to optimise the VM by specifying an operating system
--os-variant is a optional parameter, but helps further optimisation of the emulator.

Nov 14, 2012

Installing CentOS 6.3 from a USB mass storage device

I've done A LOT of research on this issue and I have finally been able to create a bootable USB to use for installing CentOS.

Download the Centos DVD for your system.

(Optional: You can run the md5sum command on your download and compare the hash against that stored on the server)
Clear the USB (NOTE: This is assuming your device is sdb!!! Double check, otherwise you may wipe your hard-drive!!!):

sudo dd if=/dev/zero of=/dev/sdb bs=512 count=1
Make it bootable (you can type in 'm' to show a help menu):

sudo fdisk /dev/sdb
>n
>p
>1
>(default)
>(default)
>a
>1
>t
>c
>w
Format the partition:

sudo mkfs.vfat /dev/sdb1
Download the livecd bash script and make it executable:

wget http://git.fedorahosted.org/cgit/livecd/plain/tools/livecd-iso-to-disk.sh
chmod +x livecd-iso-to-disk.sh
Install the software required by the script:

sudo apt-get install isomd5sum syslinux extlinux
Run the script:

sudo ./livecd-iso-to-disk.sh [your-dvd-iso] /dev/sdb1
Insert your USB device and run!

Jul 11, 2012

Ubuntu 12.04, LAMP and CodeIgniter 2

This guide aims to install and use CodeIgniter on an Ubuntu machine. We will assume you have the default Ubuntu installation.

Install Apache 2:
sudo apt-get install apache2
Install MySQL:
sudo apt-get install mysql-server mysql-client
Install PHP5 and the necessary libraries:
sudo apt-get install php5-cli php5-mysql libapache2-mod-php5
Download the CodeIgniter framework:
wget http://codeigniter.com/download.php -O ~/CodeIgniter.zip
Extract the framework to the default Apache2 web directory:
sudo unzip ~/CodeIgniter.zip /var/www/
You should now be able to navigate to your extracted folder from your web-browser. Hint: you will need to open firefox to http://localhost/<name-of-folder> which you can get by executing the following command:
ls /var/www

References

'Installing the LAMP stack for Apache2' from www.howtoforge.com
Another LAMP stack for Ubuntu tutorial from smashingweb.info
Settin up LAMP and CodeIgniter from caravaggisto.wordpress.com

Jul 8, 2012

Securing Ubuntu 12.04

I have been extended my knowledge of Linux System Administration (in particular, securing Linux systems), and as such the first thing I did when I installed Ubuntu 12.04 LTS was to lock it down.

Installing Ubuntu is outside the scope of this blog post, so if you don't have Ubuntu yet I suggest you follow the official documentation.

Networking

We will first deal with locking down our machines network access. By restricting how our machine communicates with others we narrow down the attack vectors available.

The first thing we should do is turn on a firewall. Ubuntu comes with ufw pre-installed so we will just use that (I have covered this in a previous blog post).
sudo ufw enable
Check its status with:
sudo ufw allow ssh
Enable any services you will need. For instance a web server will need the HTTP port of 80 open.
sudo ufw allow ssh

# You can specify a port directly
sudo ufw allow 80
# You can also specify whether it is TCP or UDP
sudo ufw allow 80/tcp
# Finally, you can specify whether it is incoming or outgoing
sudo ufw allow in 80
sudo reject out 1337
Sysctl allows you to configure the Linux kernel during runtime. We will edit the file /etc/sysctl to harden our network interface; open the file in your favorite editor and make the following changes:
#IP spoofing/forging protection by turning on the reverse path filter
net.ipv4.conf.all.rp_filter=1
net.ipv4.conf.default.rp_filter=1

# Protect against ICMP attacks
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.icmp_ignore_bogus_error_responses=1

# Turn off IPv4 features that are easy to abuse
net.ipv4.conf.all.accept_source_route=0
net.ipv6.conf.all.accept_source_route=0
net.ipv4.conf.default.accept_source_route=0
net.ipv6.conf.default.accept_source_route=0
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.default.send_redirects=0
net.ipv4.conf.all.accept_redirects=0
net.ipv6.conf.all.accept_redirects=0
net.ipv4.conf.all.secure_redirects=0

# Block SYN attacks
net.ipv4.tcp_syncookies=1
net.ipv4.tcp_max_syn_backlog=2048
net.ipv4.tcp_synack_retries=2
net.ipv4.tcp_syn_retries=2

# Log Martians
net.ipv4.conf.all.log_martians=1

# Ignore directed ICMP pings
net.ipv4.icmp_echo_ignore_all=1

# Don't perform IP forwarding
net.ipv4.ip_forward=0

#####
# IPv6
#####

# Number of router solicitations to send until assume no routers present
net.ipv6.conf.default.router_solicitations=0

# Do not accept router preferences
net.ipv6.conf.default.accept_ra_rtr_pref=0

# Do not accept prefix info from router
net.ipv6.conf.default.accept_ra_pinfo=0

# Do not accept Hop limit settings from router
net.ipv6.conf.default.accept_ra_defrtr=0
Reload sysctl with your changes:
sudo sysctl -p
Secure your TCP Wrapper by editing the /etc/hosts.deny file, ensuring the following line is the only one uncommented:
ALL: ALL
Allow your TCP Wrapper services (like SSH) by editing the /etc/hosts.allow file. The basic syntax is:
<service>: <host/network>
Prevent IP Spoofing via DNS by editing the file /etc/host.conf and adding the following lines:
order bind,hosts
nospoof on
If you have not already done so, update your system so that there are no security vulnerabilities:
sudo apt-get update
sudo apt-get upgrade
Install nmap, a tool for network discovery and security auditing:
sudo apt-get install nmap
Perform a local nmap TCP scan of your machine and ensure that all ports that are open are supposed to be open.
sudo nmap -v -sT localhost
Perform a SYN scan, which is another way a hacker can probe your system:
sudo nmap -v -sS localhost
Perform a UDP scan to determine which UDP services are operational:

sudo nmap -v -sU localhost
Perform the same NMap tests but on another host. If you followed the above instructions you may want to add -PN to the command so that nmap ignores the fact that your machine does not respond to pings. Note that this scan may take some time...

Filesystem

We will now protect our file-system.

Note: you will get the best security by putting your system directories into their own partition. This will allow you to specify the mount options for each directory. This guide will only cover the default install.

Protect your shared memory by editing /etc/fstab as follows:
tmpfs /dev/shm tmpfs defaults,noexec,nosuid 0 0
Bind /var/tmp to /tmp so that we limited what applications can do with that system directory. Edit /etc/fstab as follows:
/tmp /var/tmp none rw,noexec,nosuid,nodev,bind 0 0

Startup Applications

We will now modify the start-up applications and services that turn on during boot.

Display the hidden start-up applications:
sudo sed -i 's/NoDisplay=true/NoDisplay=false/g' /etc/xdg/autostart/*.desktop
Press the windows key on your keyboard, type in 'Startup Applications' and launch the program of the same name
Disable the following services (Note: These may change depending on your personal situation):

Backup monitor
Bluetooth manager
Chat
Desktop Sharing
Gwibber
Orca Screen Reader
Personal File Sharing
Ubuntu One

Disable Guest Login

Just edit /etc/lightdm/lightdm.conf and add the following line:

allow-guest=false

References:

'How to secure Ubuntu 12.04 LTS server' by thefanclub.co.za
'Securing a Linux VPS' from wolfpaw.co.uk
NSA guide to securing RHEL 5
NSA hardening tips for RHEL 5
NSA guides to securing other operating systems
'Tuning the Linux kernel', a paper by Long Yi and James Connan
'Linux Kernel /etc/sysctl.conf Security Hardening' from www.cyberciti.biz
'Security auditing with NMAP' by www.techrepublic.com
Nmap reference for scanning techniques
'Securing an Ubuntu Server' by www.andrewault.net
Ubuntu 12.04 LTS security by rationally paranoid
Modify Ubuntu blog

Feb 19, 2018

Jul 24, 2017

Jul 4, 2014

Jul 3, 2014

Nov 30, 2013

Print Screen Key

References:

Nov 4, 2013

References:

Nov 3, 2013

References

Nov 1, 2013

References:

Aug 25, 2013

Correct drivers - make sure they are installed!

Input devices

Custom Xephyr script

Xorg.conf settings

MDM configuration

Resources

Aug 23, 2013

Market Research, Wire-framing and Design

Mobile

HTML / CSS / Javascript

Deployment, DNS and Custom Domains

Aug 16, 2013

Lecture 1:

Lecture 3 & 4:

Dec 1, 2012

The server side

The client side

Further Reading:

Nov 30, 2012

Further Reading:

Nov 28, 2012

References

Nov 23, 2012

References

Nov 20, 2012

Further Reading

Nov 15, 2012

Installation

House cleaning

Creating our first guest

Further reading

Nov 14, 2012

Further reading

Jul 11, 2012

References

Jul 8, 2012

Networking

Filesystem

Startup Applications

Disable Guest Login

References: