Snap! Websites

Snap! LogoThis is the project page of the Snap! Websites CMS. It is here to give you basic information about the project. This website as a whole represents the Snap! Websites project and includes all the details you need.

At some point the source code will be made available on SourceForge.net so you will be able to compile it on your own system. Note that it has many dependencies, some of which are not available as is in your Debian or Ubuntu repository. They should appear as time passes, but at this time you will have to do a lot of manual work to get the first binary going.

Ubuntu Installation Instructions

Cassandra

Qt







 

It is now possible to install Snap! Websites CMS from launchpad using our Snap! C++ PPA (packages). By installing all of our packages you get a clean installation with all dependencies automatically included so that's the recommended installation procedure under Ubuntu servers.

Note: I uploaded a small script that you can follow to do the install. You should run the script with caution, you may want to only use it as a model, actually. See other instructions further below this page.

New Instructions

From the PPA, you want to install the snapmanager package. This allows you to then install bundles instead of trying to install the correct packages one by one on each computer of your cluster.

sudo apt-add-repository ppa:snapcpp/ppa
sudo apt-get install snapmanager

When you install snapmanager, you automatically get all of its dependencies installed so there is nearly no need to get things working. Only get the website front end to work and you will be on your Merry Way.

The snapmanager is accessible at the URL you specifically define for that website manager. For example, say you own example.com and you want to setup the manager under manager.example.com, you define the DNS to point to the server where you just installed snapmanager, and then you go to that URL using HTTPS as the protocol.

The page will ask you to accept the SSL certificate (by default we use a snakeoil certificate, you can change the setup to use letsencrypt or some other mechanism to avoid the warnings that generates by default.) Say Yes. Then you will see a list of servers. At first, just one, of course. Click on that link. The first tab (self) lists bundles that you can install to further complete you serer installation. This is the new way of installing Snap! Websites on your servers. This is extremely practical when you have to install the same things over and over again on many computers (however, at this time, managers only correctly on their own server. You can view other computers from any computer, but attempting to change parameters is likely to fail cross computers. "We're still working on it.")

All On One Server

First we describe an installation where everything gets installed on a single server. Snap! is built to work in a cluster but it can also be used on a single server (Note that Cassandra wants at least 2Gb to run and another 2Gb of system buffering, so on a single system, 6Gb or more is strongly recommanded.) Comments are added below to show what could be installed on different computers.

Basic Requirements

For Snap! to functions you'll want to complete your system with Apache2 if you don't already have it installed.

sudo apt-get install apache2

In a cluster, for security reasons, you should install Apache2 only on front end servers. It is one less thing to have to maintained on the other servers.

Note: the snapcgi is the only part that expects Apache2 to be installed, and it does not force you to install it (it is just a recommandation.)

Installation Preparations

In order for the rest of the installation to function, you want to install the following sources in your system:

Cassandra Sources (Apache.org)

WARNING: This is not the one we're currently using. Also the bundles will install all that you need so you don't need to handle this one by hand (unless you're a developer and then you probably want to do quite a bit of that by hand to know where you stand with your cassandra cluster.)

sudo add-apt-repository 'deb http://www.apache.org/dist/cassandra/debian 20x main'
#sudo add-apt-repository 'deb-src http://www.apache.org/dist/cassandra/debian 20x main'
gpg --keyserver pgp.mit.edu --recv-keys F758CE318D77295D
gpg --export --armor F758CE318D77295D | sudo apt-key add -
gpg --keyserver pgp.mit.edu --recv-keys 2B5C1B00
gpg --export --armor 2B5C1B00 | sudo apt-key add -

Note: the deb-src addition fails, somehow.

If installation a full Cassandra cluster, note that Cassandra should only be on "Cassandra Nodes". Also, once installed using the Cassandra repository, you want to look into updating the settings before you go on. By default Cassandra runs locally.

Snap! Websites Sources (PPA)

sudo add-apt-repository ppa:snapcpp

This PPA is required on the front end to install and maintain the snap.cgi binary.

It is also required on the backend servers running snapserver or snapbackend.

It is not necessary on the Cassandra nodes themselves.

Installation

sudo apt-get update
sudo apt-get install cassandra snapserver snapcgi

The update is required if you just installed new apt-get sources which are not yet known by the system. It generally takes a little while to update everything.

The install command installs cassandra, snapserver, and snapcgi and all their dependencies. After that, you are ready to run Snap!, although you need to do a lot of setup first... That the next point.

Note: Here I show you how to install all on one computer. Remember that you can install everything in a cluster, meaning that each service can be installed on a separate computer. So install cassandra on backends that do not otherwise have access to the Internet, access snapserver on other backends, again these do not have direct access to the Internet, finanlly, install snapcgi along with Apache2 on front end computers. Increase the number of computers depending on your system load. Also you can look into getting an elastic cloud system for the snapcgi/Apache2 pair and the snapserver. However, for Cassandra, this is not a good idea. It's a database and it is not really elastic.

Setting Up Snap!

IMPORTANT NOTE: we now use systemd to run daemons and the snapmanager.cgi is used to control the installation.

In order for Snap! to function as expected, one has to edit the various configuration files found under /etc/snapwebsites. Especially, you want to enter the IP address of the computer running your Cassandra cluster. Right now we only support one IP. We will soon add support to connect to any one of your Cassandra computer.

It will be necessary to restart the Snap! servers for the changes to be taken in account. The service command can be used for this purpose:

service snapserver stop
service snapserver start

Initialization of the Snap! Database

IMPORTANT NOTE: with the new snapmanager.cgi front end, this is done by the Cassandra bundle. You wn't have to do anything by hand. Just enter the correct information in the interface and click Save.

Once you have the Snap! packages properly installed, you want to initialize the database.

In order to do so, you have to run the snapserver deamon by itself. Note that by default the snapinit tool will start the server and backend tools all at once. That will prevent the first initialization process from working properly, especially on slow installations (i.e. one computer, slow I/O, etc.)

You may start the server with -d to see the output in your console (assuming you setup the logger to also show logs in the console.) Otherwise, it is simply:

/usr/bin/snapserver

Once the server is running, start the snapmanager tool. This tool has an option to start the initialization process. It can be run remotely (i.e. from your computer at home to initialize a Snap! server at work.)

/usr/bin/snapmanager

Once the tool started, make sure you enter the correct information to connect to the correct snapserver and correct Cassandra cluster. Use the Test and Connect buttons to verify that the manager can create a connection. If so, click on Tools » Initialize Website. This option opens a new window. In that new window, enter the URL to your website and then the port to be used (the default is 80. If you setup your website for HTTPS, then use 443. You may of course have setup Apache2 to listen to a different port in which case you will have to use that port instead.)

Once the URL was entered, click on the Send Request button. The process takes less than a minute to complete at time of writing. It will grow as we offer more and more plugins.

We are hoping, however, that this process will be fully automated soon.

Compiling Requirements

Straight Forward Dependencies

From a fresh installation of a 64 bit server (Ubuntu 16.04 at this time) we run the following script to get all the necessary dependencies installed. This is much easier than trying to get the list right on this website:

sudo snapcpp/bin/snap-ubuntu-packages --optional

sudo is required to run the apt-get install commands.

The --optional command line option is used to install some extras that are not required to do a build but nice to have for a complete development environment. Mainly, this includes Doxygen and additional test libraries and tools.

Note: we do not currently have any plans of supporting much more than one version, at least not until we start getting some real traction from others. So you are likely to have to move to a newer version of the

Building Snap! Websites the first time

Once all of the dependencies were installed, you are ready to start the build process. By default, the script we offer builds the debug version under a directory named BUILD. It also prepares a RELEASE directory, but it doesn't build it.

snapcpp/bin/build-snap --processors 2

This script is NOT run as root (so don't use sudo with it.)

It understands a few command line options, such as the --processors. Here is a brief help:

--processors <number> -- the number of processors you want to use to run the build. The more the merrier as long as you have enough memory and good hard drives. We suggest that you use the total number of processors minus 2 or so, if you want the computer to still be usable while it compiles.

--help -- prints out a help screen to remind you of these command line options.

--email <email> -- if you plan to have a build that generates packages, you will need to provide an email address to the various Debian/Ubuntu build commands. This is that email address. It generally needs to be yours since it will be linked to a key used to sign the packages.

--sanitize <options> -- add command line options to the g++ compiler for sanitization purposes. This is useful if you want to run such a build. It's really only if you are going to run a debug version. This won't work in production because the sanitize options make your binaries allocate 256Mb of memory up front (which is how they can make sure you don't mess your memory up...) It's virtual memory, but when you do that 100 times, it tends to make your system not so happy (at least those that don't have 2Tb of RAM.) The <option> part must one a compatible <style> as defined by "-fsanitize=<style>" in g++.

Compiling the RELEASE Version

If you'd like to compile the RELEASE version, from the same folder do:

make -C RELEASE

This build that folder. You can start the build from any folder as long as you point to the right RELEASE directory. So say you are under contrib/libtdl, you would use:

make -C ../../../RELEASE

to specify the correct folder.

Additional Dependency offered by Snap! C++ (Ubuntu 13.10 and older only)

Note that we also make use of gjslint to make sure that our JavaScript code is as clean as possible (if gjslint doesn't pass, then compilation is aborted.)

However, the Google JavaScript Linter (or Closure Linter) is not available by default on Ubuntu. For that reason we created a package for it. For you to be able to compile Snap! Websites you must have it installed on your system. You may go to the Google Closure Linter page and follow their instructions, or use our snapcpp package as follow:

sudo add-apt-repository ppa:snapcpp
sudo apt-get update
sudo apt-get install closurelinter

After that installation the snapwebsites library and plugins will build as expected.

Note that the gjslint we use is part of the snapcpp source, however, just compiling the source will not automatically install gjslint.

Compilation Failed but no errors were printed?!

Whenever you run the Makefile generated by cmake in the root directory, the resulting configuration and compilation results are saved in log files. You can find them under your build directory. Try this command to fine the files:

ls BUILD/project-that-failed

Then use an editor, more, or less to look at the files. The configuration files are named something like:

<project-name>_configure.cmd
<project-name>_configure.err
<project-name>_configure.log

In most cases, the file you are interested in is the one with the .err extension.

Similarly, you have compilation files that are named something like this:

<project-name>_make.err
<project-name>_make.log

Again, in most cases, the file you are interested in is the one with the .err extension.

Generating All the Packages

WARNING: In order to get this to generate clean packages, make sure to start with a clean directory tree. If you have development files in your existing tree, these files will end up in the source packages and the result won't be exactly what you'd expect.

It is possible to generate all the packages using make (at least under Ubuntu). However, that build process requires you to enter your password (assuming you are building as you and not as root.) This is because pbuilder does a chroot and you need to be root to do that.

make debuild     # Build source packages with changelogs updated
make pbuilder    # Expect 'enter password' as required

The build process takes a while as we make use of a quite a few of our projects and recompile them all here. It can be that the sudo times out so you may have to re-enter your password. You could extend your sudo sessions, but that is not advisable. If you mind that, you may want to look into doing that work using a virtual version of Ubuntu.

The result of this process is the same as what the PPA would do, only you can run it on any version of Ubuntu you have.

Note that this process updates your changelog files, so they will get a version that is not compatible with what we have in our own git. We will eventually find a better way to handle this at a later time.

PBUILDER Configuration for Local Packages

It is important that you first add a hook to your local pbuilder configuration so that the package dependencies are available to the chroot's apt installation as each package is built.

An excellent resource can be found here on how to set up your pbuilder configuration. Follow the instructions for "How to include local packages in the build." , but make the following exceptions: You must set up your HOOKDIR and BINDMOUNTS properly. I recommend pointing your BINDMOUNTS to your result folder. For example, if you are building "saucy" packages, point to "/home/username/pbuilder/saucy_result" (where username is your *nix account name). Also, for some reason, "OTHERMIRROR" fails to make it into your chroot apt sources.list. So, use the following modification to your hook file (I called mine D05update-local-repo):

LOCAL_REPO="/home/username/pbuilder/saucy_result"
(cd ${LOCAL_REPO} ; apt-ftparchive packages . > Packages)
echo "deb [trusted=yes] file://${LOCAL_REPO} ./" >> /etc/apt/sources.list
apt-get update

NOTE: The "[trusted=yes]" is very important in the apt line so that way it won't get upset about package signing if you don't have the local keys in the keyring.

Now when pbuilder runs, it will automatically update your package archive so packages higher in the dependency chain have their lower dependencies satisfied transparently.

With regard to optimization, you may wish to consult this document at the Ubuntu Wiki, which is an excellent resource to speeding up the compilation and package depency scan.

Requirements

This project requires:

External Libraries and Tools

  • Cassandra database system (2.0.1 or better, although it should work with older versions since 0.8.x)
  • Qt 4.8 or better (although older version may work too)
  • dl to load plug-ins
  • zipios to compress and decompress zip files [included in snapcpp/contrib]
  • log4cplus to handle logs [included in snapcpp/contrib, we use a more recent version than the one available in Ubuntu]

Internal Libraries and Tools

  • csspp, a C++ library and a command line tool to preprocess and compress CSS code
  • iplock, to block and unblock IPs in your firewall
  • libcassvalue, an interface to the Cassandra database system (content of a cell)
  • libcasswrapper, an interface to the Cassandra database system (to send CQL commands)
  • libexcep, to handle exception including stack trace from the location of the exception
  • libQtSerialization, a Qt extension to serialize data in a forward and backward way
  • TLD library to break down a domain name from any URI
  • advgetopt to handle command line options in a C++ manner
  • snapCMakeModules to extend cmake uniformly for all our projects

For more details about all those requirements, see the Requirements page.

Download

At this point, we consider ourselves at an alpha development stage. Much of the functionality is already in place, although Snap! is not yet usable on a day to day website CMS like a blog, it works properly for specialized projects (we manage one already). You may install it using our Launchpad PPA as specified above.

You can also access the source on GitHub Snap! C++ git repository on github,com to work on the source code.

AttachmentSize
install-snap.txt1.82 KB

Snap! Websites
An Open Source CMS System in C++

Contact Us Directly