Lives of the First World War after the freeze

[Matlock1418]

2 minutes ago, Keith_history_buff said:

I have asked the IWM to provide the missing data. They are saying it is unlikely that they can do this, for cost reasons, and anticipate a further response in a month or so to my FoI request. If they have plans to bring it across to the PDM, then this would negate the need for the data request, but there has been nothing formally communicated to indicate that the missing data will be added to the "permanent"? digital memorial. 

They have not filled me with confidence in their replies to my [many] enquiries / requests for corrections etc. - what I added earlier was today's unpromising reply!

[Matlock1418]

Just now, Keith_history_buff said:

Cumulatively, this is thousands of hours of activity that has been negated.

I know from personal experience

I told my other half - hundreds of hours, but they would concure - thousands [from me alone!]

Cumulatively ... ???

So sad :-(

[charlie962]

Thanks for replying to my question of missing info. I am only too aware of the poor quality current timeline display.

 

The lack of distinction for duplicate records is inexcusable. I spent many hours with help from the Volunteers to identify several hundred. It cannot have been rocket science to transfer this tag.

 

The loss of sub-unit detail is disgraceful. That is just the sort of info that is essential to understand where a man was in action and took a lot of work to uncover. Now, apparently, dismissed as unneccesary detail by a .......

 

Charlie

[Keith_history_buff]

Owing to the crowdsourced nature of LOTFWW,  you could see every field where you could make inputs. An "executive decision" has been made that certain fields are not lost, but are not "enabled" to appear on the webpage. If you take a download for an individual who was in "C" Company, this should be appearing in the download. Until recently, the battalion was not enabled to show up, but the less frequently sub-unit was. They should have "enabled" both to appear on the web page, really. 

[Guest]

4 hours ago, David_Underdown said:

It's the copyright library, but e-legal deposit does not cover databases as such.  I believe the web archive side may have crawled the original version of the site.

  Thank you DU-  I think I understand what this means -   is it the di-lithium crystals again, Captain?

The alternative is that it will have to be stored on your office pc/laptop.  Be prepared

   On a serious note, a lot of work and even more goodwill was put into this project and it is a poor sign of the times that IWM would effectively just forget about it now the parade is  over

 

[Matlock1418]

 

On 06/11/2019 at 15:05, Keith_history_buff said:

Owing to the crowdsourced nature of LOTFWW,  you could see every field where you could make inputs. An "executive decision" has been made that certain fields are not lost, but are not "enabled" to appear on the webpage

I know what executive action I would like to take against such a decision maker(s)!

We need all fields that have been populated by willing Volunteers/public

 

On 06/11/2019 at 15:05, Keith_history_buff said:

If you take a download for an individual who was in "C" Company, this should be appearing in the download

Am I missing something - how do you download an individual?

 

On 06/11/2019 at 15:05, Keith_history_buff said:

Until recently, the battalion was not enabled to show up, but the less frequently sub-unit was. They should have "enabled" both to appear on the web page, really. 

Battalions did turn up but not further sub-units or dates

 

a lot of work and even more goodwill was put into this project and it is a poor sign of the times that IWM would effectively just forget about it now the parade is  over

Agreed = as Keith said earlier = reflects very badly on IWM 

 

IWM = Note your reputation is being badly tarnished

 

And doesn't bode well for other crowd-sourced projects/databases etc. :-(

 

[Keith_history_buff]

6 hours ago, Matlock1418 said:

Am I missing something - how do you download an individual?

If you play around with the search results, it should be possible to limit it, so that one person only appears in the search results.
When this happens, you can then click on the CSV Export button

https://livesofthefirstworldwar.iwm.org.uk/searchlives/thomas oliver goldsmith/filter/type%3Dagent

[Keith_history_buff]

He was in "D" Company, so this should appear in the download.

[Matlock1418]

9 hours ago, Keith_history_buff said:

If you play around with the search results, it should be possible to limit it, so that one person only appears in the search results.
When this happens, you can then click on the CSV Export button

Not blaming you, but IWM ...

Did as advised for TOG = got a 2 row x 884 column spreadsheet!

Row 1 was headings

Row 2 was data.

All columns equal width so all needed opening to see data contents and still in a pretty unintelligible format that will need extreme mining to get the facts back out

 

Also did a search for him under surname of Goldsmith

This opened up columns a bit [I didn't count the number of columns but very many] and still extremely hard to use.

 

Tried to do for one of my POI = either way got exactly the same data splurge

Did same for a 2nd POI = same

I have loads of POI = Nightmare!!!

[regardless if for one or heaven forfend for 100+ POI!]

Did another search under surname of a POI - found him there as one row of 550+ rows with 1830+ columns which were barely intelligible against his name (except that I had some inside knowledge of the POI - for others = hopeless)

 

This cannot be what IWM promised for a free publicly-accessible Permanent Digital Memorial database

[or is it my IT incompetency?]

[Keith_history_buff]

On 06/11/2019 at 13:35, Matlock1418 said:

All I have had recently from IWM was:

"Thank you for your enquiry and I am sorry to hear that you are disappointed with the new Lives of the First World War website.

Since the launch of the new website, there have been reports of issues and glitches that our team are continually working on to resolve. Due to the high level of content that is held on the website, this can sometimes take a little longer to correct and therefore we appreciate your patience in this matter.

The website is currently undergoing updates which will mean that the submitted stories of your relatives will display in chronological order as originally intended. As this work is on-going I apologise that I am unable to offer an exact timeframe as to when all updates will be completed, however I hope this will offer some form of reassurance that your contribution is important to us and is being attended to.

Please accept our apologies for how you feel your contribution has been treated and we do hope that when the website updates are completed you will be impressed with the final result"

 

As my enquiry was not really about "submitted stories" but access to all data = Not filling me with encouragement or confidence - now/near future/ever

With regard to the comment in red, I think there has been an internal misunderstanding at IWM between who put together the response, and whoever they asked for further info.

Each story can have a date associated with it, if the contributor chose to add it. I have been told that the stories are "now" listed in the order of the date in which they were keyed into the database. That is functionality that has been there since day one, and personally I would not perceive it as an "improvement".

 

The "historical dates" are recorded in the download, but are not "enabled" to be visible on the website.

This is what is appearing for George Wroe. I have edited the screenshots, so that the "historical dates" are superimposed.

[Matlock1418]

IWM/LOTFWW/PDM muddling along as usual

 

47 minutes ago, Keith_history_buff said:

I have been told that the stories are "now" listed in the order of the date in which they were keyed into the database. That is functionality that has been there since day one, and personally I would not perceive it as an "improvement".

As before my enquiry of IWM was not strictly about "Stories" but the muddling of Stories due to date of entry is not the biggest of my concerns - at least they are showing up to the public enquirer

 

What is firstly most concerning is the lack of readily accessible and complete data sets that can be extracted - I don't know how to do it nor can get it out

Hope you may perhaps be able to help me download an understandable set of complete data for a sample individual - perhaps your George Wroe???

[Keith_history_buff]

Looking at the download, when a story was added to the database, it was allocated the next available unique identifier. It looks like about 120,000 were added, so I guess this is how the stories are presented in the sequence in which they were entered, running from left to right and then in subsequent rows where necessary.



 

[Keith_history_buff]

6 minutes ago, Matlock1418 said:

As before my enquiry of IWM was not strictly about "Stories" but the muddling of Stories due to date of entry is not the biggest of my concerns - at least they are showing up to the public enquirer

Would you be prepared to share your original question, and the IWM's response, for completeness, please?

With regard to data extracts, there are some challenges:

George Wroe's service number was 60129, so a search of "Wroe 60129" does bring up one search result limited to the entries/types of 1 for "People" entries, no "Stories" and no "Communities". From here I can get one download of his person profile, which is immense.

For stories, I think I had to do a search to capture every story including the word "Wroe" in it, so that in addition to the 11 stories relating to him, there were 5 other peoples' stories too.

The search can be limited to one of these subgroups (People, Stories, Communities) either prior to performing the search, or by applying limits to the subsequent search results.

[Matlock1418]

On a couple of more encouraging points however [from a very small sampling process]

• Timeline for an individual who had many data entries/events which were undated and yet which were placed/showing on the timeline after his death = events seems to have been brought forward into his lifetime/Timeline [even if not accurately placed - as how could that be done if undated e.g. previously a bizarre situation when his un-dated pre-war occupation was placed on the timeline after his wartime death! - sadly still now showing after he had enlisted and was a then soldier, but at least not after his wartime death as a soldier!  Often/usually? such an occupation came from his Attestation papers so could perhaps at least have been placed before a dated attestation = but at least a bit better than before.]
• Family members that previously seemed to have acquired a mysterious middle name initial to match their first forename = seem to have lost the mysterious new and inaccurate initial

 

1 hour ago, Keith_history_buff said:

Would you be prepared to share your original question, and the IWM's response, for completeness, please?

Was providing feedback / pointing out unusual situations such as the Timeline and Mysterious initial etc. [see my last post which crossed with yours] - so maybe the situation is slowly improving - so long as feedback is kept being submitted to IWM

Edited 14 November , 2019 by Matlock1418
addition

[Keith_history_buff]

Yet again, we see that the "historical dates" associated with a story have only been migrated in part. Where a "known at" date was recorded, this does appear in the download. If a "to" or "from" date was associated - i.e. "Naval Funeral at Milton" associated with his funeral on 24 September 1920 -  then this has been omitted from the data migration. 

Were it not for the printout taken from the legacy LOTFWW timeline, it would not be possible to know what specific date had been associated.

[Matlock1418]

24 minutes ago, Matlock1418 said:

What is firstly most concerning is the lack of readily accessible and complete data sets that can be extracted

As I indicated before it is not the "Stories" [stories about a person's life] that concerns me most - it is data accessibility and presentation in a meaningful way - the Excel spreadsheet download from an individual "Life Story", a.k.a. as earlier described as an individual's life 'Profile' in the thread, is not meaningful in any way in my experience

 

To give a hypothetical example for: Joe Bloggs [but which could be attributed to all LOTFWW / PDM Life Stories a.k.a. described as 'Profiles' in the thread]

Data item 1 = Forename = "Joe"

Data item 2 = Surname = "Bloggs"

Data item 3 = 

Data item 4 = 

etc. etc. etc. etc. etc. 

 

How do you pull off all the data items held against Joe Bloggs in a meaningful manner? - the Excel spreadsheet download that is possible is not meaningful

And especially if many data items are linked e.g. for Joe Bloggs = Private, #123456 Blankshire Regiment, 12th Battalion, A Company 1 Section 04-08-1914 to 11-11-1918

Along with the other often linked data information sets i.e. description and data e.g Father: Arthur Bloggs, Mother: Betty Bloggs Brother: Charlie Bloggs Wife: Daisy Bloggs [formerly Smith) etc. etc. etc. - its the etceteras that are most likely to be the potentially-extractable data that seems so disasterously missing at the moment.

 

Such a wealth of data and linked strings of data item fields were added there in the original LOTFWW data collect - How now recovered from the new PDM?

[Kath]

Can you correct errors in LOTFWW?

[Keith_history_buff]

Hi Kath,

Errors cannot be corrected, but IWM will take down material i.e. an image was shared without seeking the copyright owner's permission.

The link to Post 28 of this thread, from September 2019, talks about this a bit more

 

[Matlock1418]

18 minutes ago, Kath said:

Can you correct errors in LOTFWW?

You cannot personally now change or add any more evidence, facts, photos or stories etc.

You can submit your case to LOTFWW using one of their FAQ links if you think something is wildly wrong

See below

https://www.iwm.org.uk/projects-partnerships/lives-of-the-first-world-war/general-faqs 

Can I request something to be amended or removed from Lives of the First World War? 

We can investigate a takedown of material on Lives of the First World War in the following instances: 

1. Breach of the General Data Protection Regulation, i.e. display of personal data relating to a living individual, without the consent of that individual
2. Breach of copyright e.g. an image has been added to Lives without the copyright owner’s permission
3. Content has been added that causes offence e.g. image added to a profile that is not of that person, use of offensive language in text.

If your query meets the above criteria, please get in touch via this form. 

Unfortunately we cannot accept submissions of new information or images into the database. 

 

I used the form link above for a general query as to the state of play regarding on-going but my reply as posted above was not overly helpful

The FAQ form route was suggested for future enquiries / feedback observations etc. - but a little change has apparently been made via previous specific e-mails to the then project manager who appears to have passed them on regarding operational glitches 

Think specific issues could, and should, still be submitted and the form now seems the only realistic way [as I now have no named contact].

 

Edit - Keith - my reply just crossed with yours!

 

 

Edited 14 November , 2019 by Matlock1418
addit

[Kath]

Thank you Keith and Matlock.

I've seen FAQs.

So, no.

An error is enshrined under the aegis of the IWM

https://livesofthefirstworldwar.iwm.org.uk/community/2342

"HMT Royal Edward brought Canadian troops to Europe before being used as an internment ship for three years anchored off Southend-on-Sea in Essex. "

[Keith_history_buff]

Hi Kath, there have been errors since day one with regard to data capture. It is a given that crowdsourcing will result in one or two dubious entries being recorded. The source-based approach ought to mean that a fact can be tied back to a given source.

The IWM could spend the next 100 years trying to correct any identified "fault" with the data, and I think they have to have a cut-off.

[Matlock1418]

7 minutes ago, Kath said:

An error is enshrined under the aegis of the IWM

:-(

You could make a correction here on GWF - source-based as Keith suggested / as was the original IWM aspiration

Edited 14 November , 2019 by Matlock1418
addit

[Keith_history_buff]

That I can tell, the final change to the PDM has been to enable searches of Conscientious Objectors. Here is a screenshot of a search on "Attlee". There is no way that you can search on those doing war work i.e. munitions workers, female tram conductors, mercantile marine etc.

It can be seen there are two men named Attlee who were conscientious objectors. The filters appear on the left-hand side of the screen.

The final tasks on the IWM "To investigate" list were as follows:

• Add profile picture to media listing - completed
• Image attribution to be added - completed for non profile pictures
• Source attribution for story - completed
• Correct issue with marriage dates - not addressed for whatever reason
• Allow searching to identify and filter conscientious objectors - completed
• Enable more items to appear on a conscientious objector timeline - no idea if anything specific has been done.

I believe that duplicated images and stories have been deleted.

[Matlock1418]

2 minutes ago, Keith_history_buff said:

the final change to the PDM

Final?  :-/

[Keith_history_buff]

If you are aware of any formal communication by the IWM that further changes to the Permanent Digital Memorial will be taking place, it is news to me. Their declared aim was to down tools in the autumn, and I think this has been done belatedly by the contractor.