Larger Sites

[bhwebco]

I know I have posted a similar question before but I wanted to re-phrase it slightly.

I am currently working an a web design project and with everything I do GS is the first CMS I even consider before looking in to anything and what I am wondering is.

As far as GS goes I know its intended for smaller projects but lets say I have a site which will be adding pages consistently in a blogging style. What I did is just use i8n search and created a main blog page that pulls results for the blog tag order from most recent then under that page I created separate pages for the month and year Feb2013 etc and inside those dated pages i create a new page for each blog posting. I dont need categories or anything of that nature and wanted it to be fully searchable, metadata etc so this why I chose this method.

But what I want to know is will GS remain stable as more pages and content are added in to it such as if I end up with hundreds of pages in the system will GS start acting funny or weird? I really want to use this CMS for content driven web sites but am just trying to be safe. As it stands GS blows away most of the competition and I want to use it for more than just various micro sites or the 5 or 10 page sites.

[shawn_a]

GS supports way more than 5 or 10 pages.

[bhwebco]

Yeah I knew it supported that I was just using it as the generalized small site figure. I was just curious if GS had any limits on the amount of pages it can handle

[mvlcek]

Yeah I knew it supported that I was just using it as the generalized small site figure. I was just curious if GS had any limits on the amount of pages it can handle

Basically the GS frontend should work even with 1000+ pages quite performant, as basically independent of the number of pages only one page XML is read and output. The only parts growing with the number of pages are:

• since GS 3.1 all fields of all pages (with the exception of the content field) are cached in one file, which is read for each request, EVEN IF IT IS NOT NEEDED. If you use custom fields with I18N custom fields or I18N special pages, they are cached in ANY CASE, even if they are content like fields (e.g. for 2-column layouts). Thus the cache file might get very large.
  
• if you use queries to search the cache, the time needed will be proportional to the number of pages (but probably neglectable to the time needed for reading), but displaying the excerpts, etc. of the found pages will only depend on the number to be displayed, not the total number of pages.
  
• if you use the I18N plugin, the plugin uses its own cache file for the navigation - with only the necessary fields - which was great for pre-3.1 and would still be great, if just the GS cache used lazy loading.
  
• if you use I18N search, the indexing time grows with the number of pages - but this is just for the first user searching. However, depending on your configuration, you might run out of time (maximum execution time of PHP) or memory.
  
• searching with I18N search is of course approximately proportional to the number of pages. You might want to switch off the AJAX features (like live search).
  

On the backend:

• the main problem is the display of all pages - it will take longer for more pages. And if you don't use the I18N plugin, you won't be able to find anything in the long list.
  
• when you update pages, cache files (like the GS cache, the I18N navigation cache, the I18N search index) might be recreated (if it is not done lazy on first use) - you might have to wait a long time after saving and there is the danger of too large a PHP execution time or memory.
  

[shawn_a]

We are working on some unit test to test and find the problems, most covered above, with thousands of pages.

[bhwebco]

Thanks i feel pretty comfortable that GS will work out even if I do have to make those mentioned tweaks. I wanted a style of blogging but by no means need a featured rich ad spammy blog so I think the pages will work.

That would actually be a kind of nice plugin to have for the admin side, being able to sort pages based on an internal category so individual pages could be found easier. I do have the i8n part that allows the pages to be collapsed in tree style so I guess that is a start. Just wish I could have only that feature and not have it tamper with the Menu structure options but at this point ill take what I can get.

Is there a way to list say the 5 recently added pages titles? Or would I need to use a plugin for that?

[shovenose]

You won't have any trouble using GS for a large Site. Get-simple.info certainly doesn't have a lot of pages, but we do get a lot of traffic...

[shawn_a]

I will have a unit test to generate large amounts of pages with real content soon to test with.

[E.Eliveld]

I will have a unit test to generate large amounts of pages with real content soon to test with.

Hello,

Sorry to break in: is creating a subfolder structure for pages a solution? I'm testing GS and the lack of creating subfolders for the xml is my problem. Organizing the folders/files has become a hustle. I have a site with close to 400 different pages and I'm looking for a flat file cms that can handle much more (for years to come).

Erwin

[shawn_a]

Not really, the problem is the pages viewer, it needs better filtering capability, by tags or a new category field.
And some internals like better menu manager, and sorting caches., page index, etc

[Carlos]

is creating a subfolder structure for pages a solution? I'm testing GS and the lack of creating subfolders for the xml is my problem.

One of these days I intended to suggest that, maybe for a future GS version (4 or later, as it wouldn't work with many plugins)

[shawn_a]

How is that a problem?
Why would you use the filesystem for organizing anything ?
It is just the xml datastore, organization should be in the cms.

[E.Eliveld]

Not really, the problem is the pages viewer, it needs better filtering capability, by tags or a new category field.
And some internals like better menu manager, and sorting caches., page index, etc

Filtering in the Page Manager is based only on page title. is'nt? Is there another way?

[shawn_a]

We would have to add it, but yeah you could probably filter by tags, which would be much more helpful.

[Carlos]

How is that a problem?
Why would you use the filesystem for organizing anything ?
It is just the xml datastore, organization should be in the cms.

Why not? It's just a possible way to do it.
You even considered it... };-)

This would allow having several pages with the same slug (at different folders):

slug = data/pages/slug.xml
folder/slug = data/pages/folder/slug.xml
folder/subfolder/slug = data/pages/folder/subfolder/slug.xml

Actually, GS uses the filesystem (but it allows just one level).

[shawn_a]

Yeah as a hacky kludgy way, which is my point.
We do not need users moving files around and stuff, like I said, xml files should be considered as a datastore. If someone wants a file based system hierarchy, then why use a cms.

Wrong solution, and would introduce more limitations at some point.
Also recursing in to folders to scan files is more expensive.

The proper solution would be to use pageindex for slugs and storage.
Slugs become an alias for pageindex.
And files are stored as unique pageids.

Then slugs can become multileveled.

Code:

slug = 1.xml [slug:'slug']
path/slug = 2.xml [slug:'path/slug']
path/path/slug = 3.xml [slug:'path/path/slug']

[Carlos]

That's the approach that other flat file CMS uses.

But I (and I think I'm not the only one) always liked about GS that you (the webmaster or admin, not a user) can easily locate a page (by its slug) and edit it, not only from the backend, but simply with a text editor, without having to edit some index file too.

[shawn_a]

Who edits raw XML files?

[shovenose]

Penguins.

[Carlos]

@shovenose
Exactly.

@shawn_a
Being able to view/edit/access xml data files (users, pages) is the GS equivalent of e.g. using phpmyadmin in wordpress.

[shawn_a]

I am only taking about pages

A user could really mess things up its all encoded.
Why not just use a source editor on the website.

[Carlos]

Ok, forget user data files. Let's stick to page data files.

As I said I'm not talking about normal users editing xml files, but administrators/webmasters/advanced users being able to directly edit/fix any field (not only the content), low level, like you'd do with phpmyadmin if it was mysql-based.

This is something that we've always been able to do with GS, and one of its strong points, IMO.

[mvlcek]

...
This would allow having several pages with the same slug (at different folders):

slug = data/pages/slug.xml
folder/slug = data/pages/folder/slug.xml
folder/subfolder/slug = data/pages/folder/subfolder/slug.xml

...

My comments on this topic:

• each page needs a unique ID, otherwise managing/referencing pages becomes much more difficult - and it's a good practice, too.
  
• if each page's unique ID is also the file name - and you would need this for fast access - you don't need (and want) directories
  
• as GetSimple automatically adds a counter on page creation, if the slug already exists, this should be no problem for users
  

As long as the file name is the slug, there is no use to storing the slug as field in the XML, too.

I even think that the structure should be totally separate from the pages. If there was no need for backward compatibility, I'd remove parent and order from the page files and use instead a 'navigation.xml' file detailing the structure. You could even have multiple navigation files.

On another topic, loading the page cache should definitely be lazy - I would rather see multiple cache files for the various functions, like a navigation cache for the menu relevant fields. That way the performance would be nearly independent of the number of pages (on very large sites not all pages will be in the menu, but they would be linked from other pages).

Just my opinion to these topics :-)

[shawn_a]

3.3 will have a menu cache

[Carlos]

@mvlcek

In the multifolder structure I suggest (for some future GS release or another hypothetical flatfile CMS), the Unique ID would not be the slug, but path+slug.

The non-fancy url structure could be: SITE/index.php?id=folder/subfolder/slug
(I've used the 'id' parameter, but could be changed to 'p' or other)
You could have, say, thousands of pages across many folders. Caching? Just the minimal necessary data; not global; lazy loading...

I'm talking only about the url structure. I also think menu/navigation structure should be completely separated from that. Even now, I may want to have a multilevel menu where some of the top level items link to subpages (e.g. SITE/shop/products/offers).

BTW, nice idea about having different navigation structures (I had also thought about this, even my first script for GS was a rudimentary way to that).