Sitemap Generation & Pinging

[badcat]

I'd mentioned elsewhere on this forum or privately to Chris...

I think there needs to be a separation of Sitemap Creation and Pinging. Especially since there's no real user control over the ping - uhm nor is it "undoable"

Case in point: I just found several of my demo test sites (which I'd assumed to be private by virtue of the fact that they weren't publicly linked) showing up in Google Search results. Not so bad unless a client decides to Google themselves and winds up viewing work in progress, or possibly worse

[Zegnåt]

Then again, need they really be separated?

Is there anything besides search engines that need a sitemap? If there isn’t, then there is no reason to generate one unless you want to show up on the search engines radar. When you want a sitemap to make it easy on search engines, then why not ping them automatically?

[internet54]

I agree with badcat. Even on a localhost setup, it sends out the ping.

Here is my suggestion.

1. Click Generate Sitemap.
2. Javascript Popup. "Do you want to send your sitemap to the search engines?"
3. Click yes to ping. Click no to not ping.

[Zegnåt]

internet54, maybe you can explain to me then why you would generate a sitemap on your local setup if you do not want to get it known by Google and other search engines?

[badcat]

First to note: The feature says "Generate Sitemap" - NOT "Generate Sitemap and Ping Google, et al"... Please correct me if I'm wrong but, the vast majority of sitemap creation tools don't autosubmit by default, they "create"

Further - Yes, while it is generally understood that Sitemaps are for the benefit of search engines, there's currently no facility in the GetSimple feature that allows editing or appending Sitemap metadata such as:

* Priority
* Last updated
* Which items to not index
* Additional metadata

So the point here is that while GetSimple allows me to create the sitemap, perhaps I want to submit/ping it at my schedule and AFTER I make any modifications I want.

I'm only suggesting that if it was a simple issue of separating the two features (Sitemap creation and Pinging) and allowing the user to also check a button and submit at the same time, it'd be a more powerful (and more clearly understood) tool.

[Zegnåt]

It’s not that it’s hard to separate, the question is whether it really is up to the CMS to do this or not. Maybe a line stating that the sitemap will automatically be submitted to search engines would be appropriate.

Priority, care to explain what this is? I’m not really into sitemaps.
Last updated, isn’t this there? The standard timestamp with the documents is the latest update time, so this should be added. If it’s not, I’ll make sure it is in version 2.
Which items not to index, pages that are set to be private should not show on the sitemap. If this is not the case, I’ll check this for version 2 as well.
Additional metadata, what’s left? Descriptions? GetSimple 2 will already allow for per-page description settings, so these will then be included too.

Please give as much input on the sitemap as possible. We’d love to get this right on the first run of version 2!

[badcat]

Hi Zegnåt,

FYI - http://www.sitemaps.org/protocol.php

Additional metadata would be <changefreq>

Yes, I believe last updated <lastmod> etc ARE there. What I mentioned is that there's no way to EDIT these items from within GetSimple. And the "big picture" here is still the separation of Sitemap generation vs Pinging. I'm OK with what GetSimple currently creates for a sitemap - I'd much rather see effort go into other areas - I only want to suggest splitting Sitemap Gen and Pinging

thank you

[Zegnåt]

I know the point is the separation, but as GetSimple always wants to offer the most simple way to do something I don’t really understand how it would fit in this “big picture”.

Fact is, you don’t use the sitemap for anything besides search engines. The protocol website you linked tells us the file’s sole purpose is “to inform search engines about pages on their sites that are available for crawling”. So, if you don’t want search engines to get the sitemap, you don’t need one.

See what I mean when I ask why you would want to separate? I myself feel that having both generating and pinging under the same button is the easiest approach and should therefore be the way GetSimple takes care of it.

Maybe this is a verdict we should leave to Chris

[badcat]

OK - I'll drop this issue as I think my point is being misunderstood. As it stands the features is not useful to me. And yes, I do agree that in it's current state, "a line stating that the sitemap will automatically be submitted to search engines would be appropriate"

thanks.

[internet54]

Fact is, you don’t use the sitemap for anything besides search engines. The protocol website you linked tells us the file’s sole purpose is “to inform search engines about pages on their sites that are available for crawling”. So, if you don’t want search engines to get the sitemap, you don’t need one.

I use sitemaps all the time for purposes that have no relation to search engines.

For example. I will create a sitemap to gather all page links that my client has created and examine the times of which they were last edited.

I will also create a sitemap so I can get all address and then create 301 redirects for sites that don't use SEO url's.

So, for the later example, why would I want to ping a sitemap to Google if I'm going in to change the urls of the pages?

The sitemap creation and pinging MUST be separate. That is why I suggested a javascript popup button to ping or not to ping. Sometimes things can be overly easy.

[Zegnåt]

I'll drop this issue as I think my point is being misunderstood.

I understand the problem as you state it, I just feel this is no issue to the GetSimple target group. That’s why I said to leave the final choice to Chris.

Thanks internet54, that’s more of what I wanted to hear. Those are actual reasons why you might not want the sitemap to be issued to search engines. Still the sitemap protocol claims these XML sitemaps are for crawlers and the question whether GetSimple should split it up remains.

Maybe we should leave it the way it is but add a little “ping options” (much like the normally hidden page options) to the sitemap page? For those that want to choose what search engines to submit too, or just don’t select any at all to not submit anything.

Do you (badcat, internet54) think that could solve the issue?

That way a standard GetSimple installation could still be very simple and basic, people only change the ping settings when they know what they are doing.

[ccagle8]

After reading this, I agree: Pinging and Sitemap generation should be separate. Maybe we could hook into the fact that if the debug.xml file is found, do not ping... otherwise, ping them automatically.

More info on debug.xml: http://get-simple.info/theme-developer-tips