May 25, 2012

Setting up a CentOS 6.2 web server: Software and package integrity and installation

This is a follow on post from my guide to installing CentOS 6.2. We will go through some of the steps required to secure your server and get it ready for production use.

These steps will outline how to check what is installed on your system and whether your system has been compromised.
  1.  Log in as root and grab a current software list (check out my previous blog post on this topic).

    #Using RPM
    rpm -qa
    #Using yum
    yum list installed

     
  2. Check to ensure that yum is forced to check the gpg signature when installing packages. This is default behaviour in CentOS 6.2, but for the sake of completeness I have included this step. Check /etc/yum.conf and all the files in /etc/yum.repos.d/ for the following line:

    gpgcheck=1

     
  3. AIDE is an intrusion dectection environment that checks the integrity of installed packages and files. It can report on the changes to your system. Install it using yum:

    yum install aide

     
  4. It should be your priority to read and understand /etc/aide.conf and tailor it to your system. While the defaults should be adequate for most installations, you should nevertheless know what AIDE does.
     
  5. Generate the initial AIDE database (by default it will be stored as /var/lib/aide/aide.db.new.gz):

    /usr/sbin/aide --init
     
  6. Back up the database (in this example we are copying it to root's home directory):

    cp /var/lib/aide/aide.db.new.gz ~/
     
  7. Install the AIDE database:

    mv /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz
     
  8. Run a manual check:

    /usr/sbin/aide --check
     
  9. Stop the yum update daemon (we are going to write our own). If you followed the above guide the service won't even be installed, but again I am including it here for completeness.

    chkconfig yum-updatesd off
     
  10. Edit /etc/crontab by adding the following line (this will do a daily check of the system with AIDE):

    50 4 * * * root /usr/sbin/aide --check
     
  11. Add a file called update.cron in /etc/cron.weekly/ and add in the following (NOTE!: cron.weekly is run by the anacron service. Keep this in mind when you are disabling services later on in this guide).

    #!/bin/sh
    #
    # Update yum, then the rest of the system
    /usr/bin/yum -R 120 -e 0 -d 0 -y update yum
    /usr/bin/yum -R 10 -e -0 -d 0 -y update
    #
    # Save a list of software currently installed on the system
    /bin/rpm -qa > /root/`/bin/hostname -s`_software_`/bin/date +%Y%m%d`.txt
    #
    # OPTIONAL: Fix all the prelinks (otherwise you may get alot of prelink messages)
    /usr/sbin/prelink --all
    #
    # Update the AIDE database
    /usr/sbin/aide --update
    #
    # Make a back-up of the new database
    /bin/cp /var/lib/aide/aide.db.gz /root/aide.db.`/bin/date +%Y%m%d`.gz

     
  12. Make /etc/cron.weekly/update.cron executable:

    chmod 755 /etc/cron.weekly/update.cron
     

References

May 21, 2012

Setting up a CentOS 6.2 web server: Installation

CentOS is the free release version of Red Hat Linux with all the branding removed. It also does not have the support options and some of the fancy trimmings the Enterprise version offers, but it is still a solid server OS. This guide is a brief step-by-step guide in how to install CentOS 6.2 and configure it as a web-server.

Install from DVD:

  1. Boot up from your DVD (you will need to enter into the Boot menu of your computer OR edit your BIOS to do so)

  2. Select Install or upgrade an existing system from the menu

  3. If you are worried about your DVD you can choose to to test it, but this is not a necessary step so you can skip it.

  4. On the Welcome screen select 'Next'

  5. Select your language (in our case we are going for the default of 'English (English)')

  6. Select your keyboard type (in Australia we use 'U.S. English')

  7. If you are just going for a standard local hard-drive set-up then just choose the 'Basic Storage Devices' option. If you are going for something fancy (such as network storage or special drives), or you just want to disable some devices for that extra level of paranoid security then choose 'Specialized Storage Devices'.

    If you have no idea which one you should choose then just select the Basic option.

  8. Enter in the host-name of your new server (for best results you should append your domain name to the end so it works seamlessly with SSL certificates) i.e. testserver.example.com

    If you want to configure a static IP address click on the 'Configure Network' button, select the your network card (probably eth0) and enter away.

    If you are going to use DHCP, or just don't know, just hit 'Next'

  9. Select the correct timezone for you (just click a location on the map and it should select the closest one to you).

  10. Enter in an appropriate root password. Make as long and complex as possible (long sentences with mixed character types are easier to remember than jibberish strings; for instance 'My office is situated in 1234 fake street, Fakeville!')

  11. In this example we are going to go for a custom partition layout, so select 'Create Custom Layout'. If you are fine with defaults, just skip to part .

  12. Delete all existing partitions and do the following:
    • A /boot partition of about 100MB. Use the ext4 format

    • Create a LVM Physical Volume that fills up the rest of the hard-drive

    • Create a LVM Volume Group with a Physical Extent of 4MB.

    • Create LVM Logical Volumes on the Volume group as follows:

      • Swap space that is at least equal to how much RAM is in your server
      • /tmp/ should be as big as the largest file you will be manipulating (for instance, if you are copying a DVD you will need at least 4GB)
      •  /var/log and /var/log/audit are separated so that if your log system goes haywire it does not kill the space for other applications. Dedicate a couple of gigabytes to each.
      • /home/ and /usr/ should be a few gigabytes each. /usr/ just holds your applications and should remain pretty static, while /home/ is where you will store your personal files.
      • /var/ and /var/www/ will contain the majority of space on your system. MySQL stores your database files in /var/lib/mysql/, while Apache runs from /var/www/. Dedicate adequate space to each folder.
      • Your root folder (/) will only need a few GB of space. It will mainly hold configuration files.

  13. The system will take some time to format your hard-drive. Once it is complete it will ask you to install the boot-loader. While the defaults are suitable, for extra security you should consider password protecting your boot-loader.

  14. We can now select our packages. You can customize the system to suit your needs, but for the basics just select 'Basic Server' from the menu and the 'Customize now' from the radio buttons. Hit 'Next'.

  15. Do the following edits:
    • Base system - Remove 'Java Platform' and 'Directory Client'
    • Web Server - Add 'Web Server' and 'PHP support'

  16. Reboot your system!

References

May 19, 2012

Subject notes for 48270: Engineering Enterprise

These are my subject notes for the UTS subject 'Engineering Enterprise', which used to be a core subject for the Bachelor Engineering degree. Hopefully someone else can get use out of them.

The Business Plan

  • A written business plan shows stakeholders where the business is going and how to get there. It also analyses the internal and external environments to determine the competitive edge.
  • Opportunity based entrepreneurship are driven by the founder's personal interests. They are usually found in the high technological and business services sectors.
  • Necessity based entrepreneurship are driven because no other avenues for work can be found. They are usually found in the consumer driven (restaurants) and social services (security) sectors.
  • There are three avenues of product development:
    1. Creator develops a new transformational science or technology that is brought to market.
    2. Integrator acquires core products and combine them to develop value-added products and services.
    3. Adopter acquires products and tailors them to local needs.

Strategic Analysis

  • The competitive analysis of a business involves:
    1. Suppliers: Are there enough suppliers to meet current and future demand? Are all the products roughly the same? Are they easy to substitute?
    2. Substitute: Is it easy to switch to a substitute product? Is the substitute market growing? Does the substitute have an advantage over our product?
    3. Entrants: Is it easy to enter in our market? Are there economies of scale or huge start-up capital required? Are there knowledge or regulatory requirements?
    4. Customers: Are the customers close by? Do they need the product? Have they already got a similar product, and if so will they be willing to switch?
    5. Competitors:How much market share do the competitors already have? Can the market support another business? How competitive is the pricing? 
  • Pareto's 80/20 rule states that 20% of your customers will provide 80% of the revenue. It is vital that the business identifies these ideal customers and concentrate on meeting their needs.
  • Demographics are customer characteristics that are measurable and objective. They include demographic population size, composition (age, sex, race) and geographic location.
  • Pyschographics are more important as it aims to determine the personal values and attitudes of the market. It classifies customers according to social class, personality type and lifestyle.
  • The environmental analysis involves:
    1. Political influences includes government and industry regulations, restrictions and standards that must be followed.
    2. Economic influences involve determining whether economic indicators such as inflation, interest rates, unemployment, GDP growth, and market seasonality are favourable. 
    3. Societal influences includes population, cultural values, geographical density, and concerns of the populace.
    4. Technological influences involve determining whether the pace of innovation and technological convergence is favourable.
  • The internal value chain is:
    Purchase -> Inbound logistics -> Operations -> Distribution logistics -> Sales -> Services
  • The external value chain is:
    Suppliers -> Business -> Customers
  • Vertical integration is where one business acquires another in the same market. It increases the risk because the business is now heavily invested in the same industry.
  • Horizontal integration is where the business purchases a supplier or customer business.
  • The SWOT analysis takes into account:
    1. Strengths: What are the major competencies of the business compared to competitors?
    2. Weaknesses: What are the weaknesses of the business compared to competitors?
    3. Opportunities: What existing products can be extended to new markets? What new products do the existing market need? Is there an opportunity to bring a new product to a new market?
    4. Threats: Are the prices for supplier products increasing? Are there new regulations or technologies that threaten the existing business? Is the customer base shrinking?
  • It is important that a business excels in one or more major competencies. They can be:
    • Features: Range and distinctiveness
    • Innovation: Development of new products
    • Quality: Regularly and reliably meets customer needs
    • Intellectual capital: Patents and IP
    • Financial resources: Lots of money to spend
    • Distribution: Reach customers where ever they are
    • Service: Customer issues are dealt with efficiently and effectively
    • Efficiency: Operation is efficient compared to competitors
    • Brand: High brand recognition
    • Price: Competitive price compared to competitors
  • Types of market opportunities are:
    • Market penetration (Existing Market, Existing Product). Easiest to get into.
    • Market development (New Market, Existing Product)
    • Product development (Existing Market, New Product)
    • Diversification (New Market, New Product). The riskiest opportunity.

Direction

  • The mission statement is the summary of the business as it stands now. It is generally derive from information in the business plan. The mission statement answers the following questions:
    • Who are our customers?
    • What are the customers' needs?
    • How do we solve the need?
  • The vision statement aims to predict the future position of the business. It is where the business wants to be. It answers the following questions:
    • Who will the customers be?
    • What will their needs be?
    • How will we solve those needs?
  • Objectives are developed by converting the mission and vision statements into specific performance targets and results. They are the destination. Objectives should be SMART (Specific, Measurable, Assignable, Realistic and Time-related).
  • Strategy determines how the organisation will achieve the objectives. The strategy should be outlined in the business plan.
  • The Strategic management process can be defined as follows:
    • Analyse the internal and external environments
    • Develop the mission and vision statements
    • Set the objectives
    • Formulate the strategy and plans
    • Implement the plan through projects and business operations

Pricing

  • The pricing structure of a product can be:
    • Controlled: External entity (such as government) will offer regulated pricing or guidelines.
    • Demand: Prices reflect the value of the product to the customer. If is solves an expensive problem then the product can demand a high price.
    • Competitive: Similar products are used as a price benchmark.
    • Cost-plus: Products are priced according to how much it costs to develop plus a profit margin. Ensures that money is made but not necessarily that they will sell.
    • Premium: Products are priced higher than competitors because of perceived benefits.
  • Direct or Variable costs are those which are directly related to the amount of product produced. These are typically material and labour.
  • Indirect or fixed costs are those which are incurred regardless of how much product is produced. These include administration, utilities, R&D, and Factory costs.

Marketing and Sales

  • Key Performance Indicators (KPI) are important to measure marketing and sales. Small increases in sales conversion and repeat purchases can lead to large increases in profit.
  • A sales pipeline is used to identify individual sale opportunities for a specific product. It helps the seller sort and track the progress of 'sales opportunities', not 'accounts'.
  • A sales opportunity should be measurable ($ amount of the sale) and dated (expected date of closure). It will then undergo the following stages:
    • Suspect evaluation: A potential customer has been found and they have contacted us or we need to contact them.
    • Prospect evaluation: The prospect is evaluated in terms of demographics and psychographics.
    • Needs evaluation: Determine collaboratively what the customer needs
    • Opportunity evaluation: Determine what opportunities exist with the customer. Can we sell our product to them to solve their need?
    • Commitment: Get a solid commitment from the customer.
  • To elicit the needs of the customer, we use a combination of the following question types:
    • Background questions uncover facts about the organisation and their environment
    • Problem questions identifies problems, difficulties or dissatisfaction about the current environment
    • Implication questions help build the seriousness of the problem so that it becomes large enough to take action
    • Solution questions focus the buyer on the solution and help them develop part of it. It also reduces objections as the buyer is explaining how the solution can help them, convincing them of the value of the solution.
    • Attitude questions uncover the individual needs and serves the buyers own self-interest. These questions uncover motivations.
    • Commitment questions determine where the seller is in regards to the sale. Good questions are answerable with a yes or a no. You can determine whether the customer is ready to commit when they go from 'whether' to implement the solution to 'how' to implement it.
  • The objective of negotiation is to develop a Win-Win situation. A solution should ideally be reached through objective standards and reasoning, not threats or pressure.
  • A list of tradables should be written up, prioritised and understood in terms of flexibility. These are effectively the bargaining chips of the negotiation. Alternative solutions should be established to ensure there is no pressure to commit to an agreement.

Operations

  • The five C's of credit are:
    • Character (credit history)
    • Capacity (ability to meet credit obligations through operating cash flow)
    • Collateral (pledged assets in case of default)
    • Capital (financial reserves)
    • Conditions (general economic conditions of industry)
  • The three types of business entities are Service, Retail/Wholesale and Manufacture.
  • Types of organisational structures:
    • Simple: The founder is no longer the sole employee. Founder does managerial and critical tasks while employees perform other duties.
    • Functional: Employees do specialised roles and are partitioned into functional groups. Delegation of authority is given to functional managers.
    • Matrix: The organisation is split into line managers (for projects, products or demographic) and functional managers. Each employee has two managers to report to, with line managers handling most of the daily management tasks.
    • Divisional: Each division has it's own self-contained functional units. Each division competes for funds. Economies of scope can be created by sharing resources between divisions.

Calculations

Industry margins
Gross income = Sell Price - Cost of Goods Sold (COGS)
Gross Margin = (Sell Price - COGS) / Sell Price

Break even analysis
Revenue = Sell Price * Sell Volume
Gross Margin = (Revenue - COGS) / Revenue
Contribution Margin = (Revenue - Variable Costs) / Revenue
Break even point = 0 = Revenue - (Variable Costs + Fixed Costs)

Liquidity ratios
Working Capital = Current Assets - Current Liabilities
Current Ratio = Current Assets / Current Liabilities

May 17, 2012

Using RPM

RPM is a based software management system for Red Hat based systems such as Red Hat Enterprise Linux, CentOS, Scientific Linux and Fedora. It can refer to both the command-line utility and the file format used to contain the compiled software source for distribution. If you wish to learn more about the RPM system, I suggest you read Edward C. Bailey's excellent Maximum RPM.

  • Installing a downloaded .rpm file:

    rpm -ihv filename.rpm
  • Updating to the latest .rpm file when an application is already installed (to ensure no files are overwritten):

    rpm -Uhv filename.rpm
  • Querying for an installed package with a certain filename (\* is a wildcard character):

    rpm -qa filename\*
  • Remove an installed .rpm file:

    rpm -e filename
  • Remove multiple installed applications that have similar names:

    rpm -qa filename\* | xargs rpm -e
  • To get even more detailed information about an .rpm file we use:

    rpm -qpi filename.rpm

Apr 19, 2012

Extended Jquery tutorial

This tutorial is an extension of the basic JQuery tutorial. We will not cover the same stuff here, so if you want a quick and easy primer on JQuery please click on that link. Instead, this offers some extra background information that help put the code examples into context.

You might also want to check out my other posts such as the JQuery Slideshow or AJAX with JQuery and PHP.

What is JQuery?

JQuery is a client-side JavaScript library that can do a wide variety of tasks. This includes browser effects, AJAX, and interact with the DOM. Despite the introduction of CSS3 (which will elegantly handle browser effects and limited DOM manipulation), JQuery still has alot to offer the web developer. It has also been around for longer which means, for the moment, it has better cross-browser support.

JQuery is not meant to be used as an application framework. It does not provide a blueprint around which a developer can build a website. Instead, it offers a wide set of tools to make development much, MUCH easier.

Getting started...

Getting started with jQuery is easy; download the library script from the website and include it in your web-application as you would any other JavaScript file. Alternatively, Google provides live hosting of the jQuery so you could link to that to save on your bandwidth.

<!-- Include the JQuery Library code... JQuery will not work without this -->
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.0/jquery.min.js"></script>


Selectors

JQuery works best by manipulating the DOM (the fancy, technical name given to the HTML page displayed by the browser), but to do this it needs a way of finding the elements to manipulate. To do this jQuery sports a CSS selector engine, meaning you just use standard CSS nomenclature to select an element.

For instance, if we had a div element with an id set to foo, in CSS we would write #foo {} to declare the style for that id. In JQuery, to search for and manipulate element foo you would write:

$('#foo').function()

Make sure the DOM is ready!

A common problem new developers have with jQuery is they start querying the DOM before it is ready. Browsers sometimes execute JavaScript before they have completely parsed the DOM, and this leads to errors where you manipulate the #foo class but it has not loaded on the page yet.

This problem is solved by calling the jQuery function that waits until the DOM has completely loaded:

$(document).ready(function {
      $('#foo').css('colour', 'red');
});

References:

Apr 12, 2012

Designing relational databases

While I was introduced to database theory in University, we never really did much in the way of practically implementing that theory. As such, when I am tasked to design a database for a web application I had no idea where to start. Well, not really; I had a vague idea of where to begin.

Thus I scrambled through my notes and various on-line articles to relearn the topic. Here are my notes; use them wisely...

  • The Relational Database Model describes a database that has a series of unordered relations (or a related set of information) that can be manipulated using operations that return tables.
  • Some interchangeable terms:
    Relations = tables (set of related information)
    Attributes = columns/fields (Descriptor of the information)
    Tuples = rows/record (Actual data)
  • Each table should only represent one (and only one) thing or event.
  • A primary key is a column/attribute of a relation/table that guarantees the uniqueness of a record/row/tuple. You can only have one primary key per relation.
  • A foreign key is an attribute used to reference a primary key in another relation/table.
  •  Normalisation is the process of simplifying the design of a database so that it achieves optimum structure.

    1. First normal form: All attributes must be atomic. Attributes should not be able to be broken down or repeated in a relation.

      In other words, an attribute cannot be an aggregate or list of data.
    2. Second normal form: Every non-key attribute must be dependant on the primary key.

      In other words, the postcode attribute will be dependant on the address and not the person. It stands that the address should be moved to it's own table.
    3. Third normal form: All non-key attributes are mutually independent. And attribute should not be reproducible or rely on information already existing in the tuple.

      For instance, the total cost for a sale is the sum of all costs, and does not deserve it's own table or attribute.
  • A join is used to collate relations into one relation with relevant data. This can be used to perform other operations without going back and forth between the tables.
  • A Natural join relies on an intersection of attributes (i.e. both tables contain attributes with the same name). This can be either a primary key or a foreign key.
  • An Inner Join specifies which matching attributes to use (this is done through the USING keyword). In other words, you get to to specify which attributes are going to be used to match records together.
  • An Outer Join is used where you want to display all data, whether they have a matching record or not. This is because the inner join would simply discard any record that did not have a matching attribute in the other table.
  • A view (also called a named query or stored query) is simply a SELECT statement that is given a name and is stored in the database.
  • A good practice in database design is to avoid having NULL values stored in your relation. If the attribute is not required to make the relation complete, do we need the attribute? Or can we move it to it's own relation?
  • Some common mistakes in database design are:
    1. Poor design/Planning
    2. Ignoring normalisation
    3. Poor naming standards
    4. Lack of documentation
    5. One table to hold them all (a sort of repeat of the 2nd normalisation rule)
    6. Using identity/guid columns as your only key
    7. Not using SQL facilities to protect data integrity
    8. Not using stored procedures
    9. Trying to build generic objects
    10. Lack of testing

References

Apr 5, 2012

CSS tips and tricks

As someone with a background in developing electronics and embedded systems, it is no shock that when it comes to pretty UX interfaces I suck. My interfaces is the definition of 'total suckage'. But nevertheless, the job market demands that I at least attempt to make pretty interfaces for clients; the critical criteria for modern web applications seem to lie in the appearance instead of the substance. But alas, I digress...

I have been doing research into CSS as it is the dominant method of creating nice structured web interfaces (yes, JavaScript can do that too but I prefer to stick to using JS for behavioural coding). So I have come across a HEAP of useful tutorials that will help turn your steaming pile of crap into glistening gold!

Rounded Borders

Rounded borders looks awesome on anything. Here is the code:

.roundBorder
{
     border: 2px solid #505050;
     /* Rounded Border */
    -webkit-border-radius: 8px; /* Saf3-4, iOS 1-3.2, Android ≤1.6 */
    -moz-border-radius: 8px; /* FF1-3.6 */
    border-radius: 8px; /* Opera 10.5, IE9, Saf5, Chrome, FF4, iOS 4, Android 2.1+ */

    /* useful if you don't want a bg color from leaking outside the border: */
    -moz-background-clip: padding; -webkit-background-clip: padding-box; background-clip: padding-box;
}


Background Gradient

You can read more about this here, but for now here is some code for you to play with:

.gradient
{
    /* Background Gradient */
    background-color: #69aef1;
    background-image: -webkit-gradient(linear, left top, left bottom, color-stop(0%, #69aef1), color-stop(100%, #ffffff));
    background-image: -webkit-linear-gradient(top, #69aef1, #fff); /* Chrome 10+, Saf5.1+, iOS 5+ */
    background-image: -moz-linear-gradient(top, #69aef1, #fff); /* FF3.6 */
    background-image: -ms-linear-gradient(top, #69aef1, #fff); /* IE10 */
    background-image: -o-linear-gradient(top, #69aef1, #fff); /* Opera 11.10+ */
    background-image: linear-gradient(to bottom, #69aef1, #fff);
}


Word Wrapping/Breaking

This piece of CSS should provide nicer looking word-breaking/hyphenation:

    /* CSS3 word break */
    -ms-word-break: break-all;
    word-break: break-all;
    word-break: break-word;
    -webkit-hyphens: auto;
    -moz-hyphens: auto; 


Centre Tables (and Elements)

I used this code to make a table centre properly in one of my elements. I'm sure it could find uses elsewhere though:

.center
{
      /* Reset margins for the elements (defaults to centered( */
    margin-left:auto;
    margin-right:auto;
}


CSS toggled accordion

This effect is created using a simple unordered list with added radio button functionality. Here is the code:

/* Clean up the lists styles */
ul.accordion {
    list-style: none;
    margin: 0;
    padding: 0;
}

/* Hide the radio buttons */
/* These are what allow us to toggle content panes */
ul.accordion label + input[type='radio'] {
    display: none;
}

/* Give each content pane some styles */
ul.accordion li {
    background-color: #CCCCCC;
    border-bottom: 1px solid #DDDDDD;
}

/* Make the main tab look more clickable */
ul.accordion label {
    background-color: #666666;
    color: #FFFFFF;
    display: block;
    padding: 10px;
}

ul.accordion label:hover {
    cursor: pointer;
}

/* Set up the div that will show and hide */
ul.accordion div.content {
    overflow: hidden;
    padding: 0 10px;
    display: none;
}

/* Show the content boxes when the radio buttons are checked */
ul.accordion label + input[type='radio']:checked + div.content {
    display: block;
}


So our HTML code will then be:

<ul class='accordion'>
    <li>
        <label for='cp-1'>Content pane 1</label>
        <input type='radio' name='a' id='cp-1' checked='checked'>
        <div class='content'>
            <p>text</p>
        </div>
    </li>
   
    <li>
        <label for='cp-2'>Content pane 2</label>
        <input type='radio' name='a' id='cp-2'>
        <div class='content'>
            <p>text2</p>
        </div>
    </li>
</ul>


References:

  • This website is an absolute must! 'CSS3 Please!' lists a whole heap of cross-browser compatible CSS3 techniques that you can just copy and paste into your website!
  • Blog post by Kenneth Auchenberg on how to achieve word wrapping in pure CSS
  • Blog post by Scott Granneman on how to centre a table in CSS
  • Blog post by Mike Cherim on the different types of web page layouts; fixed, fluid/liquid, and elastic.
  • Another post about the three types of web page layouts (fixed, fluid, and elastic) by Extend Studio
  • w3schools CSS reference
  • Pure CSS toggle accordian implementation by Oliver Caldwell
  • CSS Menu Maker provides a range of open source css menus
  • A list of CSS tutorials by Design Festival