Snap! Websites
An Open Source CMS System in C++
This is the project page of the Snap! Websites CMS. It is here to give you basic information about the project. This website as a whole represents the Snap! Websites project and includes all the details you need.
At some point the source code will be made available on SourceForge.net so you will be able to compile it on your own system. Note that it has many dependencies, some of which are not available as is in your Debian or Ubuntu repository. They should appear as time passes, but at this time you will have to do a lot of manual work to get the first binary going.
It is now possible to install Snap! Websites CMS from launchpad using our Snap! C++ PPA (packages). By installing all of our packages you get a clean installation with all dependencies automatically included so that's the recommended installation procedure under Ubuntu servers.
| Note: | I uploaded a small script that you can follow to do the install. You should run the script with caution, you may want to only use it as a model, actually. See other instructions further below this page. |
From the PPA, you want to install the snapmanager package. This allows you to then install bundles instead of trying to install the correct packages one by one on each computer of your cluster.
sudo apt-add-repository ppa:snapcpp/ppa sudo apt-get install snapmanager
When you install snapmanager, you automatically get all of its dependencies installed so there is nearly no need to get things working. Only get the website front end to work and you will be on your Merry Way.
The snapmanager is accessible at the URL you specifically define for that website manager. For example, say you own example.com and you want to setup the manager under manager.example.com, you define the DNS to point to the server where you just installed snapmanager, and then you go to that URL using HTTPS as the protocol.
The page will ask you to accept the SSL certificate (by default we use a snakeoil certificate, you can change the setup to use letsencrypt or some other mechanism to avoid the warnings that generates by default.) Say Yes. Then you will see a list of servers. At first, just one, of course. Click on that link. The first tab (self) lists bundles that you can install to further complete you serer installation. This is the new way of installing Snap! Websites on your servers. This is extremely practical when you have to install the same things over and over again on many computers (however, at this time, managers only correctly on their own server. You can view other computers from any computer, but attempting to change parameters is likely to fail cross computers. "We're still working on it.")
First we describe an installation where everything gets installed on a single server. Snap! is built to work in a cluster but it can also be used on a single server (Note that Cassandra wants at least 2Gb to run and another 2Gb of system buffering, so on a single system, 6Gb or more is strongly recommanded.) Comments are added below to show what could be installed on different computers.
For Snap! to functions you'll want to complete your system with Apache2 if you don't already have it installed.
sudo apt-get install apache2
In a cluster, for security reasons, you should install Apache2 only on front end servers. It is one less thing to have to maintained on the other servers.
Note: the snapcgi is the only part that expects Apache2 to be installed, and it does not force you to install it (it is just a recommandation.)
In order for the rest of the installation to function, you want to install the following sources in your system:
WARNING: This is not the one we're currently using. Also the bundles will install all that you need so you don't need to handle this one by hand (unless you're a developer and then you probably want to do quite a bit of that by hand to know where you stand with your cassandra cluster.)
sudo add-apt-repository 'deb http://www.apache.org/dist/cassandra/debian 20x main' #sudo add-apt-repository 'deb-src http://www.apache.org/dist/cassandra/debian 20x main' gpg --keyserver pgp.mit.edu --recv-keys F758CE318D77295D gpg --export --armor F758CE318D77295D | sudo apt-key add - gpg --keyserver pgp.mit.edu --recv-keys 2B5C1B00 gpg --export --armor 2B5C1B00 | sudo apt-key add -
Note: the deb-src addition fails, somehow.
If installation a full Cassandra cluster, note that Cassandra should only be on "Cassandra Nodes". Also, once installed using the Cassandra repository, you want to look into updating the settings before you go on. By default Cassandra runs locally.
sudo add-apt-repository ppa:snapcpp
This PPA is required on the front end to install and maintain the snap.cgi binary.
It is also required on the backend servers running snapserver or snapbackend.
It is not necessary on the Cassandra nodes themselves.
sudo apt-get update sudo apt-get install cassandra snapserver snapcgi
The update is required if you just installed new apt-get sources which are not yet known by the system. It generally takes a little while to update everything.
The install command installs cassandra, snapserver, and snapcgi and all their dependencies. After that, you are ready to run Snap!, although you need to do a lot of setup first... That the next point.
Note: Here I show you how to install all on one computer. Remember that you can install everything in a cluster, meaning that each service can be installed on a separate computer. So install cassandra on backends that do not otherwise have access to the Internet, access snapserver on other backends, again these do not have direct access to the Internet, finanlly, install snapcgi along with Apache2 on front end computers. Increase the number of computers depending on your system load. Also you can look into getting an elastic cloud system for the snapcgi/Apache2 pair and the snapserver. However, for Cassandra, this is not a good idea. It's a database and it is not really elastic.
IMPORTANT NOTE: we now use systemd to run daemons and the snapmanager.cgi is used to control the installation.
In order for Snap! to function as expected, one has to edit the various configuration files found under /etc/snapwebsites. Especially, you want to enter the IP address of the computer running your Cassandra cluster. Right now we only support one IP. We will soon add support to connect to any one of your Cassandra computer.
It will be necessary to restart the Snap! servers for the changes to be taken in account. The service command can be used for this purpose:
service snapserver stop service snapserver start
IMPORTANT NOTE: with the new snapmanager.cgi front end, this is done by the Cassandra bundle. You wn't have to do anything by hand. Just enter the correct information in the interface and click Save.
Once you have the Snap! packages properly installed, you want to initialize the database.
In order to do so, you have to run the snapserver deamon by itself. Note that by default the snapinit tool will start the server and backend tools all at once. That will prevent the first initialization process from working properly, especially on slow installations (i.e. one computer, slow I/O, etc.)
You may start the server with -d to see the output in your console (assuming you setup the logger to also show logs in the console.) Otherwise, it is simply:
/usr/bin/snapserver
Once the server is running, start the snapmanager tool. This tool has an option to start the initialization process. It can be run remotely (i.e. from your computer at home to initialize a Snap! server at work.)
/usr/bin/snapmanager
Once the tool started, make sure you enter the correct information to connect to the correct snapserver and correct Cassandra cluster. Use the Test and Connect buttons to verify that the manager can create a connection. If so, click on Tools » Initialize Website. This option opens a new window. In that new window, enter the URL to your website and then the port to be used (the default is 80. If you setup your website for HTTPS, then use 443. You may of course have setup Apache2 to listen to a different port in which case you will have to use that port instead.)
Once the URL was entered, click on the Send Request button. The process takes less than a minute to complete at time of writing. It will grow as we offer more and more plugins.
We are hoping, however, that this process will be fully automated soon.
From a fresh installation of a 64 bit server (Ubuntu 16.04 at this time) we run the following script to get all the necessary dependencies installed. This is much easier than trying to get the list right on this website:
sudo snapcpp/bin/snap-ubuntu-packages --optional
sudo is required to run the apt-get install commands.
The --optional command line option is used to install some extras that are not required to do a build but nice to have for a complete development environment. Mainly, this includes Doxygen and additional test libraries and tools.
Note: we do not currently have any plans of supporting much more than one version, at least not until we start getting some real traction from others. So you are likely to have to move to a newer version of the
Once all of the dependencies were installed, you are ready to start the build process. By default, the script we offer builds the debug version under a directory named BUILD. It also prepares a RELEASE directory, but it doesn't build it.
snapcpp/bin/build-snap --processors 2
This script is NOT run as root (so don't use sudo with it.)
It understands a few command line options, such as the --processors. Here is a brief help:
--processors <number> -- the number of processors you want to use to run the build. The more the merrier as long as you have enough memory and good hard drives. We suggest that you use the total number of processors minus 2 or so, if you want the computer to still be usable while it compiles.
--help -- prints out a help screen to remind you of these command line options.
--email <email> -- if you plan to have a build that generates packages, you will need to provide an email address to the various Debian/Ubuntu build commands. This is that email address. It generally needs to be yours since it will be linked to a key used to sign the packages.
--sanitize <options> -- add command line options to the g++ compiler for sanitization purposes. This is useful if you want to run such a build. It's really only if you are going to run a debug version. This won't work in production because the sanitize options make your binaries allocate 256Mb of memory up front (which is how they can make sure you don't mess your memory up...) It's virtual memory, but when you do that 100 times, it tends to make your system not so happy (at least those that don't have 2Tb of RAM.) The <option> part must one a compatible <style> as defined by "-fsanitize=<style>" in g++.
If you'd like to compile the RELEASE version, from the same folder do:
make -C RELEASE
This build that folder. You can start the build from any folder as long as you point to the right RELEASE directory. So say you are under contrib/libtdl, you would use:
make -C ../../../RELEASE
to specify the correct folder.
Note that we also make use of gjslint to make sure that our JavaScript code is as clean as possible (if gjslint doesn't pass, then compilation is aborted.)
However, the Google JavaScript Linter (or Closure Linter) is not available by default on Ubuntu. For that reason we created a package for it. For you to be able to compile Snap! Websites you must have it installed on your system. You may go to the Google Closure Linter page and follow their instructions, or use our snapcpp package as follow:
sudo add-apt-repository ppa:snapcpp sudo apt-get update sudo apt-get install closurelinter
After that installation the snapwebsites library and plugins will build as expected.
Note that the gjslint we use is part of the snapcpp source, however, just compiling the source will not automatically install gjslint.
Whenever you run the Makefile generated by cmake in the root directory, the resulting configuration and compilation results are saved in log files. You can find them under your build directory. Try this command to fine the files:
ls BUILD/project-that-failed
Then use an editor, more, or less to look at the files. The configuration files are named something like:
<project-name>_configure.cmd <project-name>_configure.err <project-name>_configure.log
In most cases, the file you are interested in is the one with the .err extension.
Similarly, you have compilation files that are named something like this:
<project-name>_make.err <project-name>_make.log
Again, in most cases, the file you are interested in is the one with the .err extension.
| WARNING: | In order to get this to generate clean packages, make sure to start with a clean directory tree. If you have development files in your existing tree, these files will end up in the source packages and the result won't be exactly what you'd expect. |
It is possible to generate all the packages using make (at least under Ubuntu). However, that build process requires you to enter your password (assuming you are building as you and not as root.) This is because pbuilder does a chroot and you need to be root to do that.
make debuild # Build source packages with changelogs updated make pbuilder # Expect 'enter password' as required
The build process takes a while as we make use of a quite a few of our projects and recompile them all here. It can be that the sudo times out so you may have to re-enter your password. You could extend your sudo sessions, but that is not advisable. If you mind that, you may want to look into doing that work using a virtual version of Ubuntu.
The result of this process is the same as what the PPA would do, only you can run it on any version of Ubuntu you have.
Note that this process updates your changelog files, so they will get a version that is not compatible with what we have in our own git. We will eventually find a better way to handle this at a later time.
It is important that you first add a hook to your local pbuilder configuration so that the package dependencies are available to the chroot's apt installation as each package is built.
An excellent resource can be found here on how to set up your pbuilder configuration. Follow the instructions for "How to include local packages in the build." , but make the following exceptions: You must set up your HOOKDIR and BINDMOUNTS properly. I recommend pointing your BINDMOUNTS to your result folder. For example, if you are building "saucy" packages, point to "/home/username/pbuilder/saucy_result" (where username is your *nix account name). Also, for some reason, "OTHERMIRROR" fails to make it into your chroot apt sources.list. So, use the following modification to your hook file (I called mine D05update-local-repo):
LOCAL_REPO="/home/username/pbuilder/saucy_result"
(cd ${LOCAL_REPO} ; apt-ftparchive packages . > Packages)
echo "deb [trusted=yes] file://${LOCAL_REPO} ./" >> /etc/apt/sources.list
apt-get update
NOTE: The "[trusted=yes]" is very important in the apt line so that way it won't get upset about package signing if you don't have the local keys in the keyring.
Now when pbuilder runs, it will automatically update your package archive so packages higher in the dependency chain have their lower dependencies satisfied transparently.
With regard to optimization, you may wish to consult this document at the Ubuntu Wiki, which is an excellent resource to speeding up the compilation and package depency scan.
This project requires:
For more details about all those requirements, see the Requirements page.
At this point, we consider ourselves at an alpha development stage. Much of the functionality is already in place, although Snap! is not yet usable on a day to day website CMS like a blog, it works properly for specialized projects (we manage one already). You may install it using our Launchpad PPA as specified above.
You can also access the source on GitHub Snap! C++ git repository on github,com to work on the source code.
| Attachment | Size |
|---|---|
| install-snap.txt | 1.82 KB |
Snap! Websites
An Open Source CMS System in C++
