New CWGC search interface

[Auimfo]

I've just replied to David Stacey with the following email. I'll let you know if there's any further response,

Cheers,

Tim L.

David,

Thank you for your prompt response to my email regarding the changes to the CWGC database.

May I firstly say that having only been able to use the new system for a short time before it was removed, I had not discovered the improvements to the search functionality you have indicated. The CWGC should be congratulated for this step in the right direction but when considering the opportunity that existed to implement even greater functionality, these can only be viewed as relatively minor advancements. If I may point you in the direction of the following online search engine voluntarily created by the site owner who has provided this service free of charge, I think you will agree it demonstrates what the CWGC should be aiming to achieve in terms of search functionality.

http://www.hut-six.co.uk/cgi-bin/search14-21.php (he has also created a similar search engine for Second World War casualties)

This excellent service enables the user to search the CWGC records with a great deal more ability and a wider range of options. It must also be noted that this is purely a search engine and therefore necessarily links to the CWGC 's individual casualty details webpages to provide the relevant information at the completion of a search. However, with the CWGC's decision to curtail external individual linking, this service will be rendered absolutely useless.

I note your comment regarding permission to utilise individual links to casualty details and consider this somewhat of a naieve statement. Not only is internet linking commonly accepted, I was under the impression that according to the CWGC's Corporate Plan the aim is to promote the spirit of remembrance to the public. Now surely the CWGC can recognise that these memorial databases, search engines and other linked research is an example of this being achieved. I would have thought this kind of activity is to be congratulated and promoted by the CWGC as a shining example of your success. Granted, any website that aims to implement a 'pay-per-view' profit system to access information freely available from the CWGC's database should be policed and curtailed however I am unaware of any site that yet fits this category.

Again, I would implore you to reconsider implementing these changes until you have fully evaluated the massive implications this will have to people who are only following your lead by promoting the spirit of remembrance. There is a co-dependency that exists here and I think the CWGC needs to accept it has a greater responsibility to it's users than implementing such drastic changes based on a rather narrow survey. Might I suggest a system of 'Registered Users' be introduced. In this way people with a vested interest in the information supplied by the CWGC could freely register their details with you, at the same time perhaps outlining their aims and purpose. These same people could then be consulted regarding any proposed changes or alterations and given the opportunity to make suggestions or identify problems. Personally I think such a system would greatly benefit all parties and result in a world class service being offered by the CWGC.

[melliget]

Regardless of the read only data argument their data is not currently secure. The database / data has been created by CWGC and it is theirs. It cannot be secure if Geoff can get access to it. This he does currently, and I for one, am very grateful.

Let us imagine our bank had a database with all their deceased customer's financial, etc details on back to 1914, and a third party site was able to access that, I doubt we would want that to carry on for long. Different I know, yet similar.

Jim.

Let's just focus on this issue of "security" for a moment, especially as it appears to be one of the key drivers for change quoted by the CWGC.

I work in IT for a large international engineering company and, whilst I'm not a database administrator per se, I do administer and support systems that run on databases so know a little in this area. The fact is, the CWGC database is "read-only", as you say. Only CWGC staff and perhaps some external IT people, if they out-source (as they seem to do), would have write access and be able to update the information within the database, i.e. add new commemorations and make changes to existing ones. So when security is mentioned, there is absolutely no question of the integrity of the data being compromised. That's a fairly solid assumption - if the database is updateable by unauthorised means via the web, then the CWGC should very quickly change the technologies they're using and/or their IT people.

So we are only talking about "read" access. Yes, as in the case of your analogy with the bank database, even read access can be a security problem. On the projects that my company is involved in, where there are many different companies involved and certain types of information highly confidential, security is a very high priority. Sometimes, even letting the wrong people know a particular document exists, even though they can't read it, is a security lapse. But we don't have that situation here. I'll highlight the key difference between your bank analogy and this situation with the CWGC by asking a question: how can the information in the CWGC database be misused? As far as I can see, the answer to that is: it can't. The very reason for the CWGC website is to make available, to disseminate publicly, the information on commemorated individuals held within their database.

When the CWGC talks about security, I think it's talking about control. It doesn't want non-CWGC people, outside of their control, offering an alternative copy of their database, a la Geoff's Search Engine. But the shortcomings of the search facility on their website is probably the very reason why Geoff would have created his fantastic tool in the first place (please chip in here if I'm wrong, Geoff). Necessity is the mother of invention. I'm not sure how Geoff went about it but, if you can view a page of information, you can save it. If you save one page, you can save many pages and these things which we do manually can be automated and done very quickly by a computer using various tools. If saving one page takes 5 seconds, then saving 1.7 million pages takes 8.5 million or about 98 days. Too long? Run it on 10 computers and it would take less than a fortnight. To me, the introduction of the PDF only option is an indication that this is the CWGC's thinking.

Having said that, I can understand their fear about control but it still gets back to the key questions of why would people want to do this and how can the information be misused. In terms of size, the CWGC database is very small, a mere 1.7 million rows. In my company, that would be about the same as a small to medium sized project. As the number of columns, fields within the casualty details screen, are so few, instead of jealously guarding their data and only allowing viewing page by page, I don't know why the CWGC don't make their whole database available as a free database download, e.g. Microsoft Access, on their website for those who want it. They could retain all their rights as owner of the information and people would, of course, need to download updated versions of the database but that wouldn't be a problem. To take the load off their website, it could be made available in various mirrored public ftp sites around the world. The very fact that researchers and war historians would be using an offline copy on their own computers and not searching on the website would also make it faster for those using the website. Not that performance has been an issue on the CWGC site, at least not in my experience.

So, instead of trying to restrict access to their database, I think the CWGC should be making it more accessible. Making it available as a download or perhaps a low priced CD (wouldn't want to encourage piracy). Not only would all the requests for improved searching subside, although that still needs doing on the website, I can see a lot of other benefits for all concerned, not least the CWGC who's laudable work in commemorating those who died because of war would be greatly enhanced.

regards,

Martin

[kinnethmont]

Martin

Thanks for the detailed explanation. I have only a simple understanding of such things, but also think " control " in the way you describe is what CWGC are / were trying to implement here.

So, instead of trying to restrict access to their database, I think the CWGC should be making it more accessible. Making it available as a download or perhaps a low priced CD (wouldn't want to encourage piracy). Not only would all the requests for improved searching subside, although that still needs doing on the website, I can see a lot of other benefits for all concerned, not least the CWGC who's laudable work in commemorating those who died because of war would be greatly enhanced.

Not sure that this would be a good thing since the CWGC database is constantly changing. Geoff's current search demonstrates this. To be absolutely sure someone is missing CWGC has also to be checked since the man might have been added in recent times.

Wider use of the CWGC data by application and registration seems a reasonable idea.

[melliget]

Not sure that this would be a good thing since the CWGC database is constantly changing. Geoff's current search demonstrates this. To be absolutely sure someone is missing CWGC has also to be checked since the man might have been added in recent times.

They could make a new download available say once a month or even quarterly. It could be clearly marked as such, e.g. "CWGC Sept 2009" or "CWGC 2009 Q3", and people could similarly annotate their sources for books, websites, etc. accordingly. Having it available both as a CD and freely downloadable from the website would discourage pirates. And as you say, the website could always be used for recent additions. It wouldn't be a replacement for what they have now, just an adjunct. The CWGC would always retain the source, the master database. I'd quite happily pay for a reasonably priced official CD from the CWGC knowing the proceeds were going to a worthy cause.

Anyway, that's just an idea and probably unlikely to be adopted. But I thought it perhaps a way for the CWGC to turn around from this fiasco and end up with a postive result for all players.

regards,

Martin

[keithmroberts]

The reply from david Stacey at post 224 is the best news yet, and shows that after the initial defensive reaction there is a real possibility of finding a better way forward. I think its important to welcome this, and to provide constructive comment on the back of it.

Martin's suggestion might also be worth developing. A paid for download would not have many thousands of customers, but would be less expensive to operate than the prohibitive cost to most of buying SDGW from N&M., and would have a steady if modest flow of customers.

Keith

[melliget]

I agree, Keith, about post 224 offering a glimmer of hope.

I don't see why there would need to be a charge associated with the download but then I guess I'm used to the National Archives of Australia having war service records available online free of charge, whereas the National Archives (UK) charges a small fee. Having a charge for the dowload could tend to create a market for pirated copies. People wouldn't be tempted to pay for a pirate CD if it's freely available as a download.

regards,

Martin

[Chris_Baker]

I now have a meeting with David, date tba.

[Seadog]

Chris, that is the best bit of news to have come out of this sorry saga so far. Well Done!.

Regards

Norman

[bushfighter]

Thank you CWGC for your efforts in maintaining an effective and free information service for dead servicemens' relatives whilst operating on a restricted budget.

Thank you especially for the pleasing landscaping and maintainance of your cemeteries and memorials around the world.

Researchers never can be satisfied - that is the nature of the beast.

I personally think that your website has been taken for granted by too many people who perhaps expect too much for nothing.

The prevailing CWGC characteristic that stands out to myself is the preservation of Dignity and Respect in your actions around the world.

I applaud you CWGC, warts & all, because in an imperfect world you are one of good things that I admire.

Harry.

[kinnethmont]

Chris

Good news. As others have already said, Jaw Jaw is the way forward.

Out of all this CWGC / IT Dept may get an inkling of the dedication, time and effort many people round the world put to help them in perpetuating the memory of those who fell. That cannot be a bad thing.

Harry

I agree.

[The Incomer]

Well said Harry

Les

[Paul Reed]

I now have a meeting with David, date tba.

That's good news, Chris.

While I am as frustrated by this as anyone, I think we would all unite in our admiration of CWGC and I must say I have felt uncomfortable about some of the comments here, albeit in the minority. I suppose we have to remember this is new ground for CWGC, in relative terms. They are still deciding how to handle their public window to the world via cwgc.org and they do need the input of IT specialists and Net Gurus with a dual interest in WW1/WW2 to assist them on their way. Lets hope this sparks a new period of consultation and co-operation.

[KevinBattle]

I'm only a minor player in this, but I have total admiration for the work in maintaining the war graves that the CWGC performs.

If this revision of the CWGC database is to provide funding for the upkeep of cemeteries, gardeners, new headstones and the like, then I can understand the "trialling" of what could be a "Pay Per View" format. What it would mean, however, is that people who build databases for Towns, Areas, Regiments etc would be the ones paying out the most, the amateur and professional researchers who are ensuring that all are commemorated. If they are not the target, then who is?

I had been involved in a project some years ago to update the Rolls of Honour for the RBL in Hockley and found the CWGC site invaluable for tracing the correct details of men commemorated, and for unearthing many more that had been omitted. It continues to be my first port of call whenever family or friends enquire about a relation. I'm currently working on the Roll of Honour for Acton and want to utilise every means to find as much information regarding how and when they fell for their Country.

I now find that Geoff's Search Engine says that it will close due to the change in the CWGC. Having read here and on similar Forums many compliments about how useful Geoff's Search Engine is, and that it is more easily searchable, how is this possible when it is using the CWGC database? Surely all that the CWGC needs to do is use the same programming as used by Geoff's Search Engines?

The worrying aspect in all this is that the CWGC seem to be blundering along under the impression that there is a competitive threat to them lurking, when all most people want to do is find out as much as possible about the circumstances of any Casualty. The CWGC should be looking at not only becoming as good as Geoff's Search Engine, but offering enhancements over and above, that only they may have access to, such as the paid for personal family inscriptions on headstones, which may reveal further details to link to other family members or areas.

The CWGC have backed off (but haven't withdrawn) the intent to impose the revised database with it's session sensitive URL format. That format won't even allow you to store the new database Debt of Honour front page as a Favourite or Bookmark, so REDUCING the ease to access their records.

Unlike many others, I haven't invested years of my time and funds to ensure that the debt owed by the current generation to those that made the ultimate sacrifice can be remembered with honour and pride, but I can understand the despair of those who have, in realising that all their work has been diminished or destroyed by this intended change. I am also a bit saddened that even in setting up meetings, it is at a "Date To Be Advised" basis, which can be interpreted as making it too awkward for those able to make cogent representations on our behalf to attend.

I doubt that anything will be done to assuage our fears, all that will be done is to repair the problems they have identified, NOT the ones that have been put forward here. I can only hope that I am wrong.

[keithmroberts]

Maybe I am too much of an optimist, but the more considered response from the CWGC quoted in post 224, and the fact that Chris has a meeting with the IT director both seem positive to me. It really seems likely that the spec for the new front end was based on some narrow thinking, rather than any deeply held plan.

The initial response from CWGC quoted earlier in the thread, and the post on the CWGC website do seem to have been followed by some reflection, and that has to be good. The fact that the date is tba is just a reflection of the fact that diaries need to be checked and surely means no more than that.

Chris is surely one of those best able to articulate the needs of researchers and of those who wish to maintain links to records, while also having the IT knowledge to talk on sensible terms with an IT professional.

Lets hope that at the end of this we will see the permanent url's and maybe an advanced search option on the CWGC website that would remove the need for Geoff's search engine. Neither should be impossible, or should be seen as unreasonable requests.

Keith

[melliget]

I realise it's probably not known at this stage, prior to the various meetings that are to take place, but I think it would be good to know the scheduled re-release date when it is known. Perhaps the CWGC will announce this on their News and Events page. At this point, we don't know whether it's a matter of days, weeks or longer.

[kinnethmont]

I am also a bit saddened that even in setting up meetings, it is at a "Date To Be Advised" basis, which can be interpreted as making it too awkward for those able to make cogent representations on our behalf to attend.

I too am a bit saddened that, despite their reverting to the old system, obviously taking on board our findings and concerns and now agreeing to meet with Chris ( possibly others too ) to discuss the problems and solutions with them we are back to knocking CWGC.

In the trenches a policy of Live and let Live was sometimes in place. Give them a chance!!

I expect we would all want Peter Francis, Head of Communication at CWGC to attend the meeting with Chris. As we know, the man is on holiday and this is likely the reason for not being able to firm the date now.

[Chris_Baker]

I am also a bit saddened that even in setting up meetings, it is at a "Date To Be Advised" basis

You're reading too much into things. I only put "date tba" because we are still discussing which day it will be, within the next few.

[1RGLI]

Possibly another additional valid question to ask the CWGC about keeping direct URL Links to the CWGC site

Why do we need to keep links to the CWGC and not just use the CWGC Site itself? Well one area of research undertaken by school children and researchers is locating those killed in their own town or village or even Island, it would not be possible searching one specific site like the CWGC, as this is Nationalistic in its design, as servicemen joined various other national forces during the Great War conflict, including France, however using some of these "Volunteer" sites that use direct URL links to the CWGC also link to other Countries commemoration sites again using direct URL Links, thus bringing a holistic approach to searching for several individuals in a Town, City, Village, Island Parish etc, form one single source website, the need for the direct URL's is to avoid any casualty discrepancies, and provide additional information alongside the CWGC National databases official data, thus bringing quality information together about a single casualty from various official sources, it would make sense that part of this quality of information includes that supplied by the CWGC would it not? Should this single database be left out, it could only serve to alienate the CWGC from excellent research potential that is carried out World wide by individuals and researchers; there are many families whose ancestors served many nations including that of Great Britain.

Mark

http://www.greatwarci.net

The Channel Islands Great War Study Group

[ian turner]

The whole concept of using the CWGC site for research purposes needs to be fully embraced by the CWGC. Whilst I can allow for the lack of a fully searchable database from the begining of their originally going online increasingly, over the intervening years, research (for whatever motive - family or historical or local interest) is becoming the reason for consulting the website. And the longer in time we are removed from both the conflicts commemorated the more ''histroical'" will be the motives for so doing. I think we have long since reached that stage with the Great War, and rapidly approach it with the Second World War.

The impression of their mindset, given by what we read from CWGC, is that they are still looking at it from the point of view of the original cemetery registers, and providing information for loved ones and immediate next of kin who were grieving at their recent loss. The Great War generation has now passed, and it is those with historical interest who now research, whatever their motive might be.

As has been well said already by other posters, the very activity of this research is a means of diseminiating the CWGC endeavours. We are in the 21st Century, and change has been wrought, not least through the internet. The importance of that should be fully taken on board. I am sure Chris will be an eloquent and diplomatic presenter of our viwpoints when he meets. For which good luck, Chris.

[dfaulder]

I would very much endorse what Ian has just posted in terms of the apparent CWGC mindset.

However, an enhanced search would be useful for CWGC's original immediate "user group" - relatives searching for loved ones. Spelling of surnames is not constant, people lied about their age, enlisted under a Christian name other than the one by which relatives knew them, dates of death get forgotten or confused etc., etc.. Sometimes to find your father, grandfather, great grandfather etc., an enhanced search (such as Geoff's) is very useful.

David

[ianw]

Very glad that Chris is meeting the CWGC. I can't think of a better representative.

I hope the CWGC come clean on what their motives are. I trust that charges are not introduced and agree that the CWGC should be doing all it can to make it easier for all types of users of their database to get the information they require.

The CWGC are the trusted custodians - but not owners - of the information that was given to them in sacred trust by our forefathers. This information is the very stuff of Remembrance. They should be absolutely open-handed with this information.

[Kath]

I use Mozilla Firefox browser as I can enlarge the text of the GWF.

I can't access the CWGC Corporate plan pdf link given by acf in post 209.

I can open it with Microsoft IE.

Kath.

[Auimfo]

Hi All,

I received a reply from David Stacey to my second email. From it's brevity compared to the first, I dare say he's been a bit snowed under with emails protesting the proposed changes

Dear Tim

Thank you for your additional comments and suggestions which we will be looking at, with many others, before further action is taken.

Yours sincerely

David Stacey

Cheers,

Tim L.

[ian turner]

"Before further action is taken" at least sounds encouraging, but let's hear what Chris reports back before getting too excited!

[Seadog]

Members will have noticed that the original statement by the CWGC regarding the implementation of the "improved" search facility has now been removed from the web site at least as far as I can tell. Has anyone any idea what is going on and is there an update as to their intentions yet?

Regards

Norman