Lives of the First World War after the freeze

[GrenPen]

I am a bit late in commenting - as ever - but I see there's a bit of a push to encourage more items to be added to Lives of the First World War prior to it being "frozen" in March 2019. 

I came across this on a different thread, and I have found the same text reproduced recently.

https://www.greatwarforum.org/topic/259471-lives-of-the-first-world-war-subscription-site-closing/

 

On 23/03/2018 at 12:27, charlie962 said:

I have just received this notification.

    

 

Looks like what was basically a very good idea but spoilt by lack of resorces and 'ownership' is coming to an end. Does anyone know more ?

 

Charlie

 

Two recent postings in July 2018

The final (financial) year for LOTFWW:

http://www.1914.org/news/lives-of-the-first-world-war-the-final-year/

 

Family tree site
https://www.family-tree.co.uk/news-and-views/news/do-you-have-a-missing-piece-of-history-from-imperial-war-museums-8

IWM Annual report published online on 18 July 2018

https://www.iwm.org.uk/sites/default/files/files/2018-07/Annual Report and Accounts 2017-2018.pdf

Quote

We are now in the final year of the project. Lives of the First World War will remain live and active until 18 March 2019, when IWM will become the custodian of the millions of incredible public contributions that have been made over the past 5 years. Contributions to the project will not be accepted after this date, to allow us to create a permanent digital memorial which will always remain free and accessible online - this will continue to help to shape our understanding of the First World War now and in the future.

 

 

Here's a post on the subject of Lives of the First World War from last year, which makes for an interesting read
https://www.greatwarforum.org/topic/251687-iwm-lives-of-the-first-world-war/

 

 

 

 

 

 

 

 

 

[GrenPen]

I did sign up for this project as a beta tester. The site was clunky, and an early sign of the lack of care or support from FMP revealed itself when the WO 363 records appeared as the 1790 US Census. The premise that the data would be free to access for research was an incentive to key data in, notwithstanding the unwieldy process for entering facts.

From a commercial perspective: I got the impression that anyone hell-bent on documenting family members would be able to do it within the initial trial period, so subscribing would not be a desired course of action for many users. The poor quality of the loaded "seeding" data has been a curse on this project. I do get the impression that IWM wanted to go to bed with FMP on this. Wanting the project to go ahead at any cost would indicate that there was no project management going on, and if an SLA exists, not a great deal of "service" would be provided. ("Have you turned your computer off and on" and "Have you cleared your cookies" has been the bog standard answer to any failure.) The consequential ropy coverage has been something to deter a potential user, if their family member wasn't appearing on the database. 

In terms of this being a commercially viable, and profitable, project I think the wheels fell off a long time ago.

Having a professional background in data management, the "living" LOTFWW has not been my motivator, but the "frozen" LOTFWW when some of this buried treasure can be unearthed through more extensive database searches. There will be some excellent data on RAF Silver War Badges and POWs that will be unprecedented - admittedly the big precondition is that enough entries have been added to make this a reality. 

Now, how to access that data from March 2019 onwards?

In a previous job, when the main platform and its historic data was sunset, the application itself was decommissioned. The data tables remained in place, and an "emulator" was in place which brought back a given transaction. In a similar manner, I can imagine something similar being put in place.

The sexier concept, is to have the means to perform some complex searches which the current setup does not allow. "How many POWs captured in 1914 of the Royal Welsh Fusiliers are in the database?" "How many men of the 10th Battalion West Yorkshires who died at the Somme are in the database?" "Just how many men from Barnsley died in the First World War, and are in the database?".

Up to now, data searches tend to involve going into FMP or Ancestry, and the user is effectively interrogating one flat file data table. Lives of the First World War is made up of several data tables. Hopefully, they can do what The National Archives has done, publish their API, and allow people to interrogate data in that manner.

Otherwise, I foresee a requirement for something equivalent to Geoff Sullivan's Search Engine to come into existence.

[ForeignGong]

I started to enter some foreign awards and have had a lot of trouble with the uncommon awards. Croix de Guerre and Medaille Militaire are accepted but others I've had to ask the moderators to fix the names. Still no joy.

I've also had problems with MBE, their list of British awards include MOBE for "Medal of the Order of British Empire" and OBE for :Order of British Empire" but NO MBE.

I have emailed several times explaining the difference and how it has to go to work, had replies that they will look into it, but with no result. I think the problem is that the people running the site have no idea about military history and what goes with that.

Pity, as it could be a great benefit for the future.

 

Just my take on it.

[Guest]

1 hour ago, GrenPen said:

I did sign up for this project as a beta tester. The site was clunky, and an early sign of the lack of care or support from FMP revealed itself when the WO 363 records appeared as the 1790 US Census. The premise that the data would be free to access for research was an incentive to key data in, notwithstanding the unwieldy process for entering facts.

From a commercial perspective: I got the impression that anyone hell-bent on documenting family members would be able to do it within the initial trial period, so subscribing would not be a desired course of action for many users. The poor quality of the loaded "seeding" data has been a curse on this project. I do get the impression that IWM wanted to go to bed with FMP on this. Wanting the project to go ahead at any cost would indicate that there was no project management going on, and if an SLA exists, not a great deal of "service" would be provided. ("Have you turned your computer off and on" and "Have you cleared your cookies" has been the bog standard answer to any failure.) The consequential ropy coverage has been something to deter a potential user, if their family member wasn't appearing on the database. 

In terms of this being a commercially viable, and profitable, project I think the wheels fell off a long time ago.

Having a professional background in data management, the "living" LOTFWW has not been my motivator, but the "frozen" LOTFWW when some of this buried treasure can be unearthed through more extensive database searches. There will be some excellent data on RAF Silver War Badges and POWs that will be unprecedented - admittedly the big precondition is that enough entries have been added to make this a reality. 

Now, how to access that data from March 2019 onwards?

In a previous job, when the main platform and its historic data was sunset, the application itself was decommissioned. The data tables remained in place, and an "emulator" was in place which brought back a given transaction. In a similar manner, I can imagine something similar being put in place.

The sexier concept, is to have the means to perform some complex searches which the current setup does not allow. "How many POWs captured in 1914 of the Royal Welsh Fusiliers are in the database?" "How many men of the 10th Battalion West Yorkshires who died at the Somme are in the database?" "Just how many men from Barnsley died in the First World War, and are in the database?".

Up to now, data searches tend to involve going into FMP or Ancestry, and the user is effectively interrogating one flat file data table. Lives of the First World War is made up of several data tables. Hopefully, they can do what The National Archives has done, publish their API, and allow people to interrogate data in that manner.

Otherwise, I foresee a requirement for something equivalent to Geoff Sullivan's Search Engine to come into existence.

 

      A very informative post even for a techno-moron like myself (I can't understand "Star Trek").   I think it may be worth keeping a watching eye on CWGC  as well.   The base file for Lives is the CWGC listing and I suspect there may be some tie-in with CWGC on this- perhaps as an "add-on" or  link.

It seems a pity-Lives bit off more than it could chew but despite it's clunkiness and - I find- very poor layout, it would be a natural home for the slow accretion of all the good work (Much of which has popped up on this Forum) which could have been transferred or consolidated on Lives across the next few years as all sorts of other little and local databases give up the ghost. 

      As the centennial years end, I hope more attention will be given to how to preserve, maintain and make accessible all the good work done on the literally thousands of small projects that have been going.  It seems a shame but in our "Bread and Circuses" system of public culture, I suspect more public monies will be spent (as they have in recent years) on media-friendly  inanities than on long-term sense.  

 

    And just to add a note:  At present time, the only home for vanishing small databases appears to be FMP which has all sorts of odds-and-sods. It seems a shame that voluntary and -perhaps- grant aided projects may have their only continuation behind a pay-wall. But if that is the only choice between the information remaining "live" and the disappearance of good work, then so be it.

Edited 28 July , 2018 by Guest

[chaz]

1 hour ago, ForeignGong said:

I started to enter some foreign awards and have had a lot of trouble with the uncommon awards. Croix de Guerre and Medaille Militaire are accepted but others I've had to ask the moderators to fix the names. Still no joy.

I've also had problems with MBE, their list of British awards include MOBE for "Medal of the Order of British Empire" and OBE for :Order of British Empire" but NO MBE.

I have emailed several times explaining the difference and how it has to go to work, had replies that they will look into it, but with no result. I think the problem is that the people running the site have no idea about military history and what goes with that.

Pity, as it could be a great benefit for the future.

 

Just my take on it.

this was my comment on another topic, the boffins that design web sites have cost as their main governing priority. if a monkey could write a web page it would be employed for peanuts over any one of our knowledgeable members just covering their costs. As mentioned before, Ancestry throws up many errors even suggesting details miles away from your needs. others have said 'less is more' gets results. you put more information you know in and Ancestry then tells you to put in less!! so instead of getting the one person, you end up with 7,000+ to wade through.

the other thing is people putting the wrong information in, on ORBATS on FWR, I was chasing one of my great uncles and it ended up taking my information as an addition. the problem was As we often find, you go down one track and end up straying off down another.

as for LOTFWW why do you have to look for F(7), Fred(2) or Frederick(5)? all have their own results and none cross reference the others so in total 14 possibilities.

[GrenPen]

There's a few interesting points that got raised here, in particular to data quality

 

I've also had problems with MBE, their list of British awards include MOBE for "Medal of the Order of British Empire" and OBE for :Order of British Empire" but NO MBE.

I have emailed several times explaining the difference and how it has to go to work, had replies that they will look into it, but with no result. I think the problem is that the people running the site have no idea about military history and what goes with that.

 

I found it peculiar they have a dozen variations on DCM, yet the Long Service and Conduct Medal is nowhere to be seen on their awards list. I agree that the people who set up the site have no idea, and did not do their homework.

 

I concur with Chaz's comment about putting the wrong info in, and as above, no checking being done before info gets put in. There are in excess of 48,000 RNVR men who were seeded from the RN Depot (Reserves) Crystal Palace card index in ADM 339. All of these have been recorded as being in the Royal Naval Division which is a posting, and not a branch of the RN. On that premise, the 1500 or so RNVR at Jutland must have been transported via Dr Who and the Tardis (or the Star Ship Enterprise's transporter beam) from the trenches near Poelkapelle to the Danish coast.(At least 10,000 RNVR men never served in the Royal Naval Division but were employed on Sea Service.)

The "British Army soldiers" were seeded from TNA's medal index cards, and not every card relates to a British soldier. There are at least 50,000 cards with number errors. With the benefit of hindsight, these MICs should have been cross-referenced to the Naval & Military Press medal rolls, to allow for the elimination of men not on the campaign medal/SWB rolls. ("Don't worry about data quality, we will build, and an army of the general public and the tooth fairy will resolve all of this through crowdsourcing." Guess what, it hasn't happened.)

With regard to the cards themselves, 4,000,000 are for campaign medals and/or SWBs. They are in the file range WO 372/1 to WO 372/22. There are some cards exclusively for the Indian Army, in the file range WO 372/26 to WO 372/29, and are in excess of 20,000. There are an additional 500,000 or so cards which cover gallantry awards.

 

It was recently stated on the side of a bus, and I believe it, that 1.4 million men served in the Indian Army. From day 1 of the launch of LOTFWW, the message was that the Great War was "the white man's burden", as no Indian or South Africans were labelled as such in the database. This gets controversial, as this anomaly is "endorsed" by the IWM, and flies in the face of the IWM charter: - chiefly the vision for the IWM was that it would be so complete that every individual, man or woman, soldier, sailor, airman and civilian from across Britain and the Commonwealth would find a record of their contribution.

The 20,000 or so should have been marked as Indian Army, but this was overlooked. This could have been a quick and easy win. (I do get the impression that an important GWF member who was involved at the start has been able to make a slight difference to the format of the Indians, but management did nothing to pick up on this, and nothing can be done until the site is frozen, should a person wish to trawl for Indian Army.) Similarly, in order to fudge the numbers, a lot of those gallantry cards have resulted in half a million duplicates, and it just adds to the impression that the project was neglected and unloved.  

 

(Somewhat ironically, the 700 or so men who do appear in the database as RNVR are from the MIC feed and classed as British Army, so this "tells" me that IWM think the RNVR was a regiment in the British Army.)

 

One potentially positive element was a merger in 2014 between some CWGC data and LOTFWW. If you knew that 12345 Private John Smith of the 3rd Foot and Mouth was in the BEF, died at Langemarck and was recorded as such on both the MIC and CWGC, a link was affected. If you have a man named 56789 Thomas Atkins on his MIC, but is T Atkins on the CWGC, the link would have failed, and no remedial action was done by FMP to resolve breakages. After this date the "other" datasets were seeded. Yet again, the tooth fairy and the general public is expected to give up time to make these linkages to dead Australians, Canadians, Sailors, Airmen etc.

I have to say that I am somewhat perplexed by the "Lives lite" sites which have popped up in the past few years from CWGC, RBL and AWM to name but a few. It beggars the question as to what will happen to those sites when the centenary is over. 

Fortunately, a lot of commemorative websites, containing research on a local war memorial or similar, have signed up to be on the British Library's preservation list, so that if/when the plug is pulled, there are snapshots of the data. I have asked for my website to be added to the list. I have also set up the website in a primitive manner, so that the data can be accessed via Excel or similar, and is not behind a paywall.

 

https://beta.webarchive.org.uk/en/ukwa/collection/114?page=1

 

 

[Keith_history_buff]

Lives of the First World War is no more. It cannot be updated. The domain has temporarily disappeared.

It will be coming back in June 2019, albeit known as the Permanent Digital Memorial (PDM). If you search "PDM", prior posts on this subject will appear.

The digital media consultancy Surface Impression are building the new platform

https://surfaceimpression.digital/

 

The data is being migrated from the old platform to a new one, so a certain amount of data cleaning needs to be done, in addition to the creation of the new data archictecture.
The Sheffield-based firm Knowledge Integration is undertaking this activity. 

https://www.k-int.com/

 

 

[Keith_history_buff]

On 11/02/2019 at 09:45, Keith_history_buff said:

the long and the short of it is this:
 

1. IWM is migrating the data from the LOTFWW site to a new PDM website, which will go live in June 2018. It appears that the new website will have a similar look and feel. Whilst the tender document seems to suggest that any user can download it (all 20 gigabytes??), it is hoped that the non-graphical element of search results and communities can be downloaded, but this will need to be confirmed by IWM nearer the time. It does appear that the abysmal search facilities will be improved, but again, no formal communication of how this is to be improved is forthcoming.
2. The data table structure is being enhanced. I had hoped for there to be an API in place, so that "Geoff's Search Engine" type searches can be performed. (You can get search results based upon regiment, service number and name - the other fields cannot be interrogated). My interpretation is that this could be done in future if more funding were made available, but I do not think this will be in place in June 2019.

The concern that I have is this - FMP & IWM got it into their heads in 2014 that all they had to do was to create a website, do nothing further and millions of people would subscribe. This would be done without consulting the prospective user base. I do not see evidence of IWM having actively engaged stakeholders to seek their opinions. In terms of user requirements, use cases to describe the desired functionality (like what I wrote in blue font in the fifth post in this thread) and the like, the question I find myself asking is "What are the developers using as a base line to build the new system if there are no user requirements?" It's too reminiscent of LOTFWW in 2014 which also followed a "build and they will come" approach. 

Whilst the spectre of funding - that is to say a lack of adequate resourcing - has, in my personal opinion, always hindered the potential of this project, there should still be some user engagement, so that a wide range of requirements can be identified, and based upon the budget, it could be determined as to which can be done, and which fall out of scope owing to budgetary constraints.

The biggest surprise with the response that I got was the radio silence with regard to IWM entering an affiliate marketing arrangement with FMP, in the same manner as long long trail. If you were to look at someone interesting, and would want to see more about said person from their service record, it would make sense to know that their record is available via FMP. If the interested party were to purchase credits or a subscription, that person learns more, FMP gets a new customer, and IWM gets a finder's fee, which can be used to offset the costs of the database. This seems to me to be a win-win scenario, unless there is some sort of confidential side letter, as part of the IWM-FMP agreement whereby IWM waives any finders fees.

 

This was taken from a different thread, and summarises some information that was prised out of the IWM by using a FoI request. I don't anticipate hearing anything else about this until the new platform goes live at the end of June. That other thread goes into way more detail than what is stated here.

[Keith_history_buff]

One expression that rings true with me is that “you have to sometimes lose a battle in order to win a war.” I think that ring true with this current development leading on from LOTFWW, for reasons that I will explain.
 

Let's take the metaphor of putting a man/woman on the moon, and launching a space programme. One of the things that would be needed would be food for the astronauts. It would be a good idea to speak to the likes of Unilever, and to draw on their decades of experience in the food business. At a stretch, you could ask them to build the rocket, too. Baked beans in a metal cylinder, people in a metal cylinder, surely it's not that much of a difference, is it? Likewise, if you can keep tinned tomatoes in an airtight environment, then it can't be that much of a stretch to maintain an airtight environment for astronauts? Is it realistic for the likes of Unilever to cover all three bases?
 

To my mind, this is where LOTFWW went wrong. FMP has experience of providing access to genealogical data in static data tables. It had no experience in running websites, providing multiple functions, and the ability to handy rapidly changing dimensions in a dynamic database. It is not a systems integrator, and has no experience of on-boarding datasets, with a view to aggregating data, and making record merges where needed.
 

For those of us in the UK, we've all seen the advert “Carlsberg doesn't do holidays, but if we did...” 
 

It is a funny way of getting across the message that Carlberg is a very good product. 
 

The reality is this though – their area of competence and expertise is in brewing. It has taken thousands of man-hours of the past experience of their employees to keep and to maintain this level of proficiency. If Carlsberg had built the LOTFWW website in 2014, I do not think it would be very good.
 

There is widespread criticism of the LOTFWW being clunky. I think that the website was a best endeavour, but given that this is not an area where FMP has its core competencies, then you cannot expect it to be a success. Could FMP have turned this into a new business, or is it the case that it had this one contract only, and that whilst it was happy to have agreed to provide historic data, the one catch with the contract was the associated need to build a website and an associated dynamic database?
 

[Keith_history_buff]

To go back to the space programme allegory above, it seems fair to say
“Houston, we have a problem”. The problem is that the activities have not been properly compartmentalised, and FMP – like Unilever in the fictional example – is trying to do everything, even the areas where they do not have the requisite expertise.
 

I would like to think that lessons have been learned, or to go back to my opening lines, in losing the initial battle, the overall outcome of the war can be won.

In terms of a replacement website, IWM have tendered for a contractor with experience of building websites for museums, and reproducing user-generated content.
 

Similarly, in terms of the data integrator, IWM has a contractor with experience in structuring data, content management, and industry-specific experience on similar projects that it has successfully executed.
 

Looking at their websites (the links are 3 posts prior to this), I am impressed with what they do. If only we could turn the clock back, to have made more resources available and could have given them the responsibility to do this from day one! I also think that the contract is prestigious, so in this regard, they will ensure they go about their duties in a manner to paint themselves and IWM in a positive a manner. If they were to mess up, it could destroy their reputation, and this would pose a threat to their ongoing existence as a small business.
 

It is in everybody's interests for this to be a success, and I have a sense of optimism that is higher than before in relation to this project. 
 

[familyhistoryman]

Hi

Do we know when in June The Lives of the First World War will be available and will it contain the Conscientious Objector database?

Tony

[petwes]

On 22/06/2019 at 07:39, familyhistoryman said:

Hi

Do we know when in June The Lives of the First World War will be available and will it contain the Conscientious Objector database?

Tony

This afternoon according to their Facebook page.

100th Anniversary of treaty of Versailles

Peter

[Open Bolt]

You are right, it is there. https://livesofthefirstworldwar.iwm.org.uk/     

I'm not sure I expected my contributing username to be changed to some alphanumerical code.

Edited 28 June , 2019 by Open Bolt

[Keith_history_buff]

Some UAT feedback


 

08-May-2019
Overall layout:
Whilst the look conformed with the IWM standard, it was confusing and lacking explanation or guidance. It is not what I would call intuitive. Based on the user stories / use cases, some brief narration should allow those persons landing on the site to easily determine how their user stories can be progressed:

"I came on here to try and find the profile for my Great Grandfather. He was Thomas William Stratford, service number 5555 and was in the South Wales Borderers"
"A family member was on HMS Queen Mary at the Battle of Jutland, and I would like to see others in the same group/community"
"Would I be able to do a search and see all [99 or so] men of the Canadian Navy" 

I think the big question is just what are these user stories / use cases? 

Performing searches:
The legacy system had some filtering options for the search results, and I did not recall seeing this yesterday.

The legacy system would list search results, typically 20 individual profiles per page. One often repeated request has been for the ability to take a download of search results, or a download of a community, as a *csv, *.xml or similar format file. We did not get to ask you if this requested enhancement has been forthcoming in the prototype.

The legacy system had one search facility only. The prototype appears to have a second search, which is perhaps to allow a search among a subset? Again, this is not intuitive. My initial search for an individual failed, but worked when I then went to the main page.

In comparison with the rest of the presentation, there is a considerably smaller text that appears at the bottom, stating "7 results found" or similar. It would be better to have this appearing at the top,  

Communities:
With regard to the layout that I briefly saw of a community, I did see the following that was not ideal:
There is some descriptive text which accompanies a community/group. This appears in a narrow and elongated format, with vast amounts of white space to the right, which makes the site look clunky. It would be better to have the text to wrap across the screen.
The presentation of the legacy site would show nine thumbnails of profiles. The existing "thumbnails" take up too much space, and overwhelm the view.
Whilst the terminology "community" has been used up to now, I think "group" is a better plain English term. Somewhere on the site, there should be a landing page that explains the concepts of groups, and has some guidance/FAQs for persons to progress further in performing searches on groups etc. 


Timeline:
The unique selling point of the legacy system was the timeline. It brought together information, and the manner in which it was presented, it effectively told a tale. Whilst I have been uncomfortable with the jargon of "life story", the great timeline on the legacy system did in fact narrate a "life story". One clever aspect of the original timeline is that for those facts that were enabled to appear on the timeline, such as unit/ship, it was possible to associate an image with it, which would appear on the timeline. This was not appearing on the timeline that we tested yesterday. (Images associated with a given fact do appear on the timelines of individuals who are on "Ancestry" family trees.)

 

As well as images, "stories" would appear in the legacy timeline, and that is not appearing in neither the May 2019 prototype, nor in the Jan 2019 mockup.

 

One profile that did have a great timeline was that of 5555 Thomas William Stratford, and I looked at his profile yesterday.

 

One of the facts that I was hoping would be enabled was the innovative "age at a given date", but this did not show up. This field is useful where the actual age of a person is recorded at one point, and the declared age is recorded at another. Two of these are recorded for Thomas William Stratford.

When I looked at his timeline, and that of others, I did come across date-related errors. There would be facts with dates. There would be a number of undated facts that would subsequently appear on the timeline, undated, and this looked awful. 
  

Unfortunately, I do not have a representative image of the timeline from the legacy website. I do have images from the January 2019 mockup, which I will send across.

 

Profile / Life Story for an individual

As before, I think the term "life story" is too confusing, and a term like "profile" would be better understood by the general public, going forward.
There was some mention that "not all the data is in the test set". It would be useful to know what items have been omitted. Owing to the time constraints, I did not get to check the content, other than seeing surname, forename and medals. If you could find this out, and let the testers know specifically what is not there, this will help with the setting of expectations. 

The unit and rank table for Thomas William Stratford was chaotic, which suggested there was some issue with the date info - it seemed to have been "lost" for some of the entries.

Stories
Owing to the sheer lack of time, I was not able to spend time looking at this. As mentioned earlier, these do not appear in the timeline at present.

 

09-May-2019

 I got the impression that the biggest concern from my fellow volunteer testers was that the current prototype does not emulate the timeline in the same manner as the legacy website did.

I do have some further feedback. If there is anything here requiring clarification, please do not hesitate to get in touch.

Communities:
It only returns ten men. The summary for each man used to contain birth and death, now it contains unit and rank info. This does not look good. I am not aware of how the other men in the community can be seen, and I have no idea how many profiles are contained in a community, either.

Life Story / Profile

The facts associated with a Life Story were categorised, and were accessible via four tabs. These four tabs also appear in the January 2019 mockup. (See page 3 of 3 of PDM_pt1_Jan.pdf) All that appears in the May prototype is name details, unit and rank, and awarded medals. It is not possible to see the associated title and description with the image that is used as the profile picture. If a  Life Story / Profile has other images associated with it and appear in the "multimedia" section, the associated description can be seen.  

For soldiers, their battalion is not appearing in the prototype. The dates associated with the unit and rank are not being displayed.

Weirdly, full user names are appearing, rather than the obfuscated names that were proposed, in order to conform with GDPR concerns. This is ironic, given that source ids and not source descriptions are appearing in the timeline.

Even more scary, I called up the details for one man who is duplicated. (Paul Maze, a friend of Winston Churchill.) There is nothing to indicate that the one profile is merged, and that the active profile is elsewhere. The volunteers have spent thousands of hours identifying duplicates, and marking profiles as such. Under the legacy setup, a "duplicate" status would be visible, with a link to the "active" profile. Please can you ask what is being done to emulate this.

Stories
Just like the "Community" header, this appears in a narrow column on the left-hand side, with lots of white space to the right. Whilst I was able to associate an image to a story for George Wroe, and this appears at the top of the story page, it would be better for both to be on the timeline.

There should be a rule whereby if no associated image is displayed, then the text should wrap across the screen, as it looks like a badly put together website, otherwise. It gives the impression that something fundamental is missing.
 

Search facility
The bad news is that this is like the IWM Collections site. It would appear that by clicking on the term "agent", the search results are limited to a list of Life Story profiles. Again, some guidance on how to filter results would make for a better user experience.

It is not possible to filter on unit, just on rank, and country (but how is this populated?) so the filtering is not great in my opinion. This is a step back, not a step forward. How will the average user be able to find their relative?

It would appear that "agent" is the means by which a search results set can be filtered. It would be better to replace this with a standardised term. My recommendation would be "person profile" or similar. I would advise against the "life story" term. This is owing to the fact that "stories" are a category in their own right. Either this term remains, which seems apt, or it is replaced with something like "anecdotes".

 

Timeline

Whilst there's nothing new to report here, it's disappointing to see the timeline for George Wroe on the prototype website and see what a mess it currently is in comparison with the legacy website's timeline.

 

10-May-2019

 

There is one final observation with regard to a test that was performed, and a glitch that came to light with the functionality:

Search results:

It is possible to perform a search. The search results appear, and at the bottom of the page, there are the lists of pages of results. 

To take the example of "american expeditionary", I was able to obtain a list of the results. It states there are 144 results appearing over 8 pages. I can see the individual Life Stories / Profiles / Agents on page one. I see a new set when I click on page two. Thereafter, when I click on the different page numbers, the view does not refresh and the same listing appears.

 

18-Jun-2019

 

Hello *******,

I just wanted to let you know that I have come across an inconsistency.

I used the search facility to bring up a profile. The search parameters used were 10521 Jackson. This successfully brought up the profile of the grandfather of Peter Jackson, the famed film director of the Lord of the Rings film, and the IWM film "They Shall Not Grow Old". This latter film was dedicated to him.

 

When the search results appear with a summary of his profile, it advises that he died in 1940, yet it states that he was born in 1970. The timeline is peculiar in that the first entry is him being awarded the Distinguished Conduct Medal in 1970. The next entry is his birth in 1892.

It looks like some sort of data capture failure has occurred. If you perform a search on Home Guard, there are several profiles that appear, and most of them have been similarly affected, to present a birth date of 1970.

Hope this can be investigated and remedied.

 

Best wishes

Keith  

The PDM site is now live. I guess there are the following questions to be asked of the PDM – an emulator of the legacy LOTFWW platform, albeit using static data, and with functionality limited to the presentation of the data, and a data search capability.

 

To what extent does the emulator functionality (i) match the legacy platform (ii) fall short of the legacy platform, and (iii) improve upon the legacy platform in relation to 

1. Presentation of the textual and visual data
2. Ease of navigation around the site
3. Ability 'to access and research' by means of its search engine, and any facilities to capture and download data

I was talking to an IT Developer friend about this at a Xmas party. He said that the new portal may not be to everyone's taste, given the diversity of the user base. He did raise an optimistic point, though.
 

He did say that if the back-end data was published via an open licence, via an API, this would allow those members of the general public, with the inclination, to build their own tailored “Geoff's Search Engine”. Whilst it's good for others to use, it's especially good for a junior developer to have such handiwork in their portfolio. (I am not an  IT Developer, but this sounds plausible to me.)
 

The one fly in the ointment is just how long it will take for this - access to the data via an API or similar - to become a reality?

[charlie962]

20 minutes ago, Keith_history_buff said:

It only returns ten men.

Which is a bit frustrating for 'my' communities which sometimes have over a hundred members. Where are the others to be found? Communities was such an important feature.

Charlie

Edited 28 June , 2019 by charlie962

[Keith_history_buff]

The fact that is is now possible to download search results would seem to suggest that in the not too distant future it will be possible to download communities. They certainly need to restore the ability to navigate via the various pages of profiles associated with a given community.

[Keith_history_buff]

I see that the name of an individual appears in the search results. In the past it would show the "most recently updated" unit and rank. There was a hullaballoo from people demanding that the highest rank held should be showing, which was an unrealistic expectation of system intelligence, given the austere budget.

It is a relief to see that the glitch, whereby most of the men were born in 1970, has now been resolved.

 

Another positive is that the timeline can be aligned to either horizontal or vertical.

[Keith_history_buff]

It was formally announced two minutes ago on FB, with the following

https://www.facebook.com/LivesOfWW1/photos/a.238597462959559/1304807249671903/?type=3&theater

 

Quote

If you experience any issues, please send us a direct message on Facebook.

 

 

 

 

[Keith_history_buff]

Developments since UAT and today

Quote

08-May-2019
Performing searches:
The legacy system had some filtering options for the search results, and I did not recall seeing this yesterday.
One often repeated request has been for the ability to take a download of search results, or a download of a community, as a *csv, *.xml or similar format file. We did not get to ask you if this requested enhancement has been forthcoming in the prototype.

Rather than just a free-form search field, which was in the original test version, an advanced search result filer is now available. The scope of the search has been extended and now covers the content of the "Stories".

The presentation of the search results was in lines, with 20 entries on each page. The new platform does allow for the layout to change. The buttons allow the ability to switch between line and grid layout.

Downloads of search results in *.csv format are now available.

Quote

08-May-2019
Profile / Life Story for an individual
As before, I think the term "life story" is too confusing, and a term like "profile" would be better understood by the general public, going forward.
There was some mention that "not all the data is in the test set". It would be useful to know what items have been omitted. Owing to the time constraints, I did not get to check the content, other than seeing surname, forename and medals. If you could find this out, and let the testers know specifically what is not there, this will help with the setting of expectations. 

There seems to be the use of the plain English term “Person”, although references to “Life Story” and “Agent” also appear.

Not all data fields are "enabled" to appear on the new website, but the data for an individual can be extracted by downloading search results as a *.csv file.

 

When a person profile is called up, the image associated with that person appears on the right-hand side of the page. This format also appears when an image is associated with a person's Story or if associated with a Community.

 

Quote

09-May-2019

Communities:
It only returns ten men.
The summary for each man used to contain birth and death, now it contains unit and rank info.

The glitch is still there, but is understood to be under ongoing investigation. On the legacy platform, a grid format of 3x per row and 3x per column were appearing. fields. On the new platform, the buttons allow the ability to switch between line and grid layout.
Birth and death info for a Person is now appearing, which will be useful for filtering, especially if you have a common name, such as David Williams in the Welsh Guards!
If a Community has an image associated with it, the narrative is on the left-hand side, and the image is on the right. This does look good, but if there's no associated image, this does leave "white space" on the right-and side of the narrative.

 

Quote

08-May-2019
Timeline:

This is like Marmite - either you prefer the legacy timeline, or you prefer the new timeline. I would imagine it is here to stay, but the date issues involving the timeline will be remedied in due course.
Since the first prototype, more facts have been "enabled" to appear, which is a positive. The new platform allows the user to change the orientation of the timeline from horizontal to vertical if desired.

 

Quote

18-Jun-2019
it states that he was born in 1970. The timeline is peculiar in that the first [timeline] entry is him being awarded the Distinguished Conduct Medal in 1970. The next entry is his birth in 1892.

This bug was very quickly resolved.

[Keith_history_buff]

On 22/06/2019 at 07:39, familyhistoryman said:

Hi

Do we know when in June The Lives of the First World War will be available and will it contain the Conscientious Objector database?

Tony

The original site had access to FMP data. The names of the 16,501 individuals on the Pearce Register will have a corresponding entry, but the dataset itself can only be accessed via a FMP paywall.

The following should take you to links to a FAQ about the new static data PDM version of IWM's "Lives of the First World War" database. I've seen the link, but not accessed it yet.
https://www.facebook.com/LivesOfWW1/photos/a.238597462959559/1307134959439132/?type=3&theater

I've not done any boolean searches with ? or * characters, which used to be possible with the legacy setup. I've found that the new search does not like the forward slash character. Instead "Walter Tull 55" rather than "Walter Tull F/55" worked instead.

[charlie962]

Thanks for the update Keith,

Charlie

[Keith_history_buff]

As and when there is progress on other items, I would imagine they will either be picked up by users, who will add to this thread, or will be announced via official channels. The addition of links to two forms of documentation is a positive.

[Keith_history_buff]

A quick recap, since last week:

Fri 28 June 2019
1400hrs       It appears that the migrated site went "live" in its Production environment. In addition, an auto-direct facility was in place, so anyone clicking on a link to the old address would be redirected

 

1500hrs        Announcement on Facebook that it had gone live. A link to both the new portal, and a FAQ listing were forthcoming.

 

Mon 01 July 2019
0905hrs       Users were reminded to contact the IWM LOTFWW FB group via FB messenger, to report any issues or bugs. 

1609hrs       As well as reposting a link to the FAQs, a link to a "search guide" was also posted

Tues 02 July 2019
Out of whatever list of issues had been reported, four enhancements came into being.The deployment of the quick fixes is a positive, and it is presumed that further activity has been turned towards addressing the bigger issues

1. Search Results - This had been showing 10 Persons, now it shows 20
2. Community listing -  This had been showing 10 Persons, now it shows 100
3. Duplicates - Profiles with duplicates have the prefix of "Duplicate of" followed by a link to a person profile, which may be self-referencing
4. Source info - The source-based philosophy was that the source of data was attributed. This was not "enabled", but now, there is an "i" button next to many fields of info, so that the data can be "audited".

One other enhancement was a statement for websites in the "External Evidence" list, with the statement "IWM is not responsible for the content of external websites. Please note that some sources will require payment to view."

[Keith_history_buff]

Tues 09 July 2019
0936        Announcement on Facebook that the community glitch had been resolved. It is now possible to view pages, comprising 20 persons per page, and to navigate by clicking on the page numbers of the listings. 

It appears that a similar glitch with search results is now resolved.

[Keith_history_buff]

As per the details on other threads, part of the migration process did involve migrating the data. One consequence is that when a search takes place, it is possible to download the data as a *.csv file, and I have been looking at the downloads to get a feel for the field names.

I have seen that with FindMyPast, it has been possible to use various limits on searches using the graphical user interface. This has resulted in two outcomes. The first is that the search parameters appear as a URL, and the second is that the search results appear. (This, to me, reminds me of using the graphical user interface to build a query in MS Access or similar. In this instance, my interaction generates a SQL script rather than a URL.)

This is the URL that appears when I type "Ceylon Planters Rifle Corps" into the search
https://livesofthefirstworldwar.iwm.org.uk/searchlives/ceylon planters rifle corps

 

With this "base" subset of 258 profiles, I can click on the rank of Corporal, and the following URL appears
https://livesofthefirstworldwar.iwm.org.uk/searchlives/ceylon planters rifle corps/filter/%7B%22lifecycle.service.rank.keyword%22%3A%7B%22Corporal%22%3A%22Corporal%22%7D%7D

In essence this is
"base" subset /filter/ field + value + value
(I have no idea why the limit value has to be repeated twice.)

I have found that, by playing around, and using a spreadsheet to compile test query URLs, it is sometimes possible to generate results that have not been possible. To start with, there has to be some sort of base subset, and then further limits can be added to the URL

The one downside is that exact matches are being made. I cannot search within, say, an address field to return all instances where "Woking" is in the address. If a free form text field has been used for data population, there could be any number of variations on a spelling.

I have an interest in the South Wales Borderers, so it has been they, rather than the Ceylon Planters Rifle Corps, that I have tended to use as a base subset.

Those searches that have been successful, and interesting (to me) have been:

Men associated with the South Wales Borderers who have been awarded the 1914 Star with clasp:

https://livesofthefirstworldwar.iwm.org.uk/searchlives/south wales borderers /filter/%7B%22awards.award.keyword%22%3A%7B%221914%2BStar%2Bwith%2BClasp%22%3A%221914%2BStar%2Bwith%2BClasp%22%7D%7D 553 results, some men may have been serving with other regiments at the time Men of the South Wales Borderers who lived at an address of the exact spelling of "Chatham Barracks" - all 2 of them https://livesofthefirstworldwar.iwm.org.uk/searchlives/south wales borderers/filter/%7B%22locations.address.value.keyword%22%3A%7B%22Chatham%2BBarracks%22%3A%22Chatham%2BBarracks%22%7D%7D   Men of the South Wales Borderers who have a wife associated with them in the database - 537 examples

https://livesofthefirstworldwar.iwm.org.uk/searchlives/south wales borderers/filter/%7B%22lifecycle.marriage.spouse.@link.relationship.value.keyword%22%3A%7B%22wife%22%3A%22wife%22%7D%7D
If the base subset were amended to "South Wales Borderers Rollinson", it would only bring back those men named Rollinson in the South Wales Borderers who have a wife recorded.
To go one further, apply a base subset limit of "Rollinson" and it will return every Rollinson with a wife

Men of the South Wales Borderers who had a FMP CWGC record manually added to their profile - 517
https://livesofthefirstworldwar.iwm.org.uk/searchlives/south wales borderers/filter/%7B%22sources.title.keyword%22%3A%7B%22Commonwealth%2BWar%2BGraves%2BCommission%2B(CWGC)%2BCasualty%2BRecords%2BTranscription%22%3A%22Commonwealth%2BWar%2BGraves%2BCommission%2B(CWGC)%2BCasualty%2BRecords%2BTranscription%22%7D%7D
 

Men of the South Wales Borderers who had a FMP CWGC record added by an automated process to their profile - 5223

https://livesofthefirstworldwar.iwm.org.uk/searchlives/south wales borderers /filter/%7B%22sources.title.keyword%22%3A%7B%22Commonwealth War Graves Commission (CWGC) Casualty Records%22%3A%22Commonwealth%2BWar%2BGraves%2BCommission%2B(CWGC)%2BCasualty%2BRecords%22%7D%7D

Men of the South Wales Borderers who have a value of "1st Battalion" recorded - 2592 instances
(It will be unlikely that every man was in "1st Battalion South Wales Borderers ")
https://livesofthefirstworldwar.iwm.org.uk/searchlives/south wales borderers /filter/%7B%22lifecycle.service.unit.section.keyword%22%3A%7B%221st%2BBattalion%22%3A%221st%2BBattalion%22%7D%7D

To conclude, 37,593 soldiers have FMP British Army Service Records 1914-1920 attached to their profiles
https://livesofthefirstworldwar.iwm.org.uk/searchlives/army /filter/{"sources.title.keyword":{"British+Army+Service+Records+1914-1920+Transcription":"British+Army+Service+Records+1914-1920+Transcription"}}

I have tried to build queries that have failed with the following:
Search on men of the SWB who were injured in 1914
Search on men of the SWB who were born in 1885
(Although four digit date numbers are in the download, my querying fails.)
Search on men of the SWB who are flagged as having been a POW 

I hope others out there can benefit from this, and build something similar for their bespoke needs.

REMINDER: The download is limited at present to 500 line items