Operation War Diary launch, 14th January 2014

[Guest]

On the 26th of January, I reported a blurred image of a page in the 1st Wilts War Diary. I received a re-scanned image of the page on the 28th of March.

Mike

[spof]

Matthew

It is still possible to download diaries to your laptop in Kew. The speeds aren't great. The most frustrating part is the 1 download at a time limit since they were moved onto the new system. I used to be able to have 2 downloading at the same time for the old diaries and other items such as the PoW interviews in WO 161.

I've been lucky in that the ones I have downloaded are the correct ones. Considering the massive (millions?) number of pages compared to the reported errors, the project generally seems to have been completed well. I suspect that like many IT projects coming up against marketing timetables, the IT QA lost out.

Glen

[David_Underdown]

It's a small team dealing with the rework (and they also have to deal with other download issues on Discovery). It's also complicated by the number of different image sets there are, the hi-res master images for long-term preservation, and different sets of lower resolution images for Discovery and Operation War Diary. A lot of cross-checking is needed to make sure the problems have been fixed everywhere (and new ones aren't introduced). In some cases things can't be completely fixed without some rescanning.

[Guest]

I had an email yesterday acknowledging the mis-catalogued 1st Bn Lincs war diary. Reported on 1st Feb, some 2 months ago. I can only assume they are swamped.

If anyone knows how to get 2nd Bn York and Lancs do let me know (6th Div, 16th Inf Bde in 1914) ...I can't get past 16th Inf Bde....

Also there are five Territorial Force battalion War Diaries that disembarked in 1914 that don't appear to have been released...

1/28th London..landed 28th Oct 1914 (WO 95/128)

1/5th Border.... landed 26th Oct 1914 nothing showing until Dec 1915 and not downloadable

1/4th Suffolk.....landed 9th Nov 1914 (WO 96/3927/2)

1/4th Seaforth...landed 7th Nov 1914 (WO 95/3941/2)

1/6th Welsh......landed 29th Oct 1914 nothing showing until Nov 1915

Not sure why they have not been released. My understanding was that the Infantry diaries had all been digitised. These are the only infantry diaries from 1914 I can't appear to download. Maybe something to do with the L of C troops or Indian Army Corps and not directly part of a British Division. The TF battalions that were attached to Regular Divisions are downloadable. Seems an unfortunate distinction given these units were five of only twenty TF battalions to land in 1914 and quite historically important.

For some reason the 5th Border Regt War Diary from 1st Feb 1918 is available online but none of the preceding months are which is a bit puzzling.

[Guest]

If anyone knows if the diaries for the Indian Army Corps have been digitised and if so when they will be released I would be interested....1st Bn HLI in 1914 and 2nd Bn Leicestershire Regt in 1914 for example

[Guest]

As far as I know none of the Indian Corps is online yet (if ever will be). The excellent HLI Chronicles are available, and do cover the 1st battalion

Mike

[Guest]

Matthew

It is still possible to download diaries to your laptop in Kew. The speeds aren't great. The most frustrating part is the 1 download at a time limit since they were moved onto the new system. I used to be able to have 2 downloading at the same time for the old diaries and other items such as the PoW interviews in WO 161.

I've been lucky in that the ones I have downloaded are the correct ones. Considering the massive (millions?) number of pages compared to the reported errors, the project generally seems to have been completed well. I suspect that like many IT projects coming up against marketing timetables, the IT QA lost out.

Glen

Glen

My experience in downloading at TNA has varied. Some months ago it was simply impossible as the link continually broke after 20 mins of downloading and the file lost. After a dozen attempts I gave up. More recently I had greater success but it is extremely slow and I suspect contention ratios are an issue. I certainly had better success on a quieter day.

I would agree that the number of files that are mis-catalogued appear to be very few as a per cent of the total. Having downloaded over 200 diaries in the past, I have only seen 2 diaries completely mis-labelled and the few mentioned on this thread. I imagine correcting these is quite labour intensive.

MG

[David_Underdown]

WO 95/128 hasn't been digitised yet, 1/28th London were GHQ troops, only divisions downwards were done in the first set of digitisation (ie Army and Corps troops and Lines of Communication are not yet digitised). The Indian Infantry Divisions have been digitised, but there's at least one more tranche of material to be released on Discovery from what's already been scanned.

For 2nd Battalion York and Lancs, it looks like the individual item shaven't actually uploaded to Discovery for some reason (a slightly different error to what's been seen before) - if you look at the "Browse by Hierarchy" view in Discovery (http://discovery.nationalarchives.gov.uk/SearchUI/browse/C4554761?v=h) it has an icon indicating it's an individual document, rather than the folder type icon used for the other pieces around it (though the description indicates it should have attached items). I'll raise this with the Discovery team, as it's probably their end of things, rather than the Digital Downloads team.

[Guest]

WO 95/128 hasn't been digitised yet, 1/28th London were GHQ troops, only divisions downwards were done in the first set of digitisation (ie Army and Corps troops and Lines of Communication are not yet digitised). The Indian Infantry Divisions have been digitised, but there's at least one more tranche of material to be released on Discovery from what's already been scanned.

For 2nd Battalion York and Lancs, it looks like the individual item shaven't actually uploaded to Discovery for some reason (a slightly different error to what's been seen before) - if you look at the "Browse by Hierarchy" view in Discovery (http://discovery.nationalarchives.gov.uk/SearchUI/browse/C4554761?v=h) it has an icon indicating it's an individual document, rather than the folder type icon used for the other pieces around it (though the description indicates it should have attached items). I'll raise this with the Discovery team, as it's probably their end of things, rather than the Digital Downloads team.

Thank you David.

Much obliged. Great pity about the Artists Rifles.

I had pinged an email to TNA ref 2nd Y&L but was not sure the flash-to-bang time on this.

As far as I know none of the Indian Corps is online yet (if ever will be). The excellent HLI Chronicles are available, and do cover the 1st battalion

Mike

Mike Thanks. I have a distinct recollection of someone telling me the Indian Army Corps diaries were being released on 10th March. Clearly not the case. MG

[spof]

Martin

This post indicates late September for Indian units.

http://1914-1918.invisionzone.com/forums/index.php?showtopic=207532&hl=digitisation

Glen

[David_Underdown]

2nd Y&L was already under investigation (presumably as a result of Martin's email), and the issue does seem to be on the Discovery side, so should be fixable aside from the other things.

The hope is still to digitise the remainder of the series before the end of the centenary, but obviously it's a matter of getting the funding.

[Guest]

Martin

This post indicates late September for Indian units.

http://1914-1918.invisionzone.com/forums/index.php?showtopic=207532&hl=digitisation

Glen

Thanks Glen.

A much ignored formation that arguably plugged the most vital of gaps when the old Regular Army had been annihilated and before the New Army was ready. Sadly only one badly written history. Very glad to see that their story will be released. Thank you.

MG

[Guest]

4th Bn Royal Fus War Diary for 1914-1915 (WO 95/1413/1); Diary claims to run from Aug 1914- Dec 1915. there are two errors:

1. It runs from Aug 1914 to Nov 1915 (Dec 1915 is missing)

2. The months are out of order. It starts with Nov 1914 then Aug 1914..Oct 1914.

Incidentally when ordering the Discovery indexing shows 1st Jan 1914 as the start date. For people experienced with WO 95 it is manageable but the layperson might be understandably confused. My initial reaction was that Aug-Oct 1914 must have been lost or I had downloaded the wrong file. All emailed to TNA but just flagging here - an example of some of the errors creeping into the production. This is a very typical example.

MG

P.S. On a more positive note having searched the diaries for the 1/28th Bn County of London Regt (Artists Rifles) (TF) I discover the (undigitied) Artists Rifles do make a guest appearance in the 1st Bn King's Shropshire Light Infantry War Diary in Dec 1914. They enter stage left on 3rd Dec 1914 and again on 8th Dec 1914 taking centre stage;

3rd Dec 1914. LA FLAMINGERIE FERME. Day passes without incident. At 7:00 pm hearing that the enemy's working parties are out repairing their wire at a pre-arranged signal each man on the parapet fire 3 rounds rapid. Lt R S WHITMORE 3rd Bn KSLI reports his arrival 8:00 pm. Two Coys of the the ARTISTS RIFLES are attached to the Battalion.

And again on 8th Dec 1914:

8th Dec 1914. LA FLAMINGERIE FERME. Large portions of the fire trench and communication trench fall in and are revetted with sacks. Three of our Pls in the firing line are relieved by Coys from the ARTISTS RIFLES

So despite being Army Troops Mars and Minerva were certainly in the line in Dec 1914.

[Guest]

I it possible TNA have just digitised them in the order they came out of the box, and they assumed that what it said on the box, was actually what was in the box.

I suppose checking every page was where it should have been, may have taken a lot longer. Just thinking out loud.

Mike

[GraemeClarke]

Hi

See my post here about the Sherwoods diary and missing bits

http://1914-1918.invisionzone.com/forums/index.php?showtopic=209391&hl=

Graeme

[Guest]

I it possible TNA have just digitised them in the order they came out of the box, and they assumed that what it said on the box, was actually what was in the box.

I suppose checking every page was where it should have been, may have taken a lot longer. Just thinking out loud.

Mike

I suspect they were simply copied in the order they were found in the box. Anyone who has taken out a box of War Diaries will know the state some of them have been left in.

My point here is one of optimising a process to save unnecessary additional labour later on: if budgets are limited perhaps some attention to detail at the beginning might save duplicating time and effort (and money) at a later date. One doesn't need specialist knowledge of Great War history to know that Nov 1914 comes after Oct 1914 and not before Aug 1914. This is not a case of reading every page, merely looking at the dates on cover sheets. A tiny amount of attention to the dates on the cover sheets might have saved more work

The small team is clearly swamped fixing an unknown quantity of issues. If it is taking two months simply to get an email acknowledgement I assume it means there must be a huge backlog. There is an asymmetric cost/benefit; Small extra attention at the beginning of the process will save a disproportionate amount of time and effort (and resources) fixing problems. this is a well understood principle of process engineering.

MG

[spof]

Mike

In 2012, the war diaries were removed from public access in stages and manually sorted by volunteers so they would have been in order. After sorting, they were made available again to read in a separate reading room and hopefully remain in the correct order. As Martin says, the state of them prior to that was horrendous so there is a chance some mixing could have occurred there.

The problems reported here are mostly either parts left out of a diary and they may well be in the next unit's collection or units being assigned the wrong Catalogue reference. While I've been going through the Catalogue for digitised diaries some of the entries state that some pieces were not catalogued correctly but that was fixed during the sorting. It is possible that some slipped through.

Martin

The delay in responding doesn't necessarily mean the TNA are fully swamped with errors from this project. They are still releasing batches of diaries along with the other documents thy deal with like the 20 year rule. I also get the impression that some Govt departments are taking this chance to hand over as many items as possible so they can reduce their own storage costs given the general level of recent budget cuts.

There is also the issue of the need to make multiple copies of the images. As David Underdown has explained elsewhere, there is the high quality archive copy plus the ones we see. While most of this is probably automated, the fact humans are involved there will be mistakes made.

Of course, they could have handed the project fully to Ancestry, or worse, Findmypast (and until this week I never thought I'd write that!!) and then we'd have more cause for complaint. I know I've been lucky with the diaries I have all being the correct ones and how frustrating it must be to get the wrong ones, but the mistakes are being resolved albeit a bit slower than most would like.

Glen

[Guest]

Thanks Glen. Trust me, I am more than happy they have been made available online. A regular visit to Kew is not an option for most of us, so here's to the team who did it. Mistakes happen, and if they can be ironed out that's fine. I think I would rather have some mistakes, than wait years for a perfect version, though it can be frustrating.

Cheers Mike

[Guest]

I see the price of a single file has changed from £3.36 to £3.30 sometime between 28th Mar and 4th Apr.

MG

[lukesmith]

Hello,

For those interested in what's going on at Operation War Diary, I've posted an update on what we've been doing since the launch on January 14th (my birthday!). Please have a read over at:

http://blog.operationwardiary.org/2014/04/08/inside-team-war-diary/

I've also taken the liberty of highlighting the Great War forum as a great place to engage in conversation about the history of the First World War, beyond the Talk forum on operation War Diary itself -- hope that's Ok with everyone here.

As it says in the blog, we haven't been idle and there's plenty more to do.

best regards,

Luke (Smith)

IWM

[kenf48]

Posted on Daily Zooniverse Facebook page today

Your map of WW1

http://daily.zooniverse.org/2014/04/24/your-map-of-ww1/

posted for information -make of it what you will

Ken

[Chris_Baker]

According to the map, British soldiers were dying at Ypres some weeks before war had been declared! There are also men dying on what appears to be the Severn Bridge near Newport ... nice animation, though.

[spof]

Chris

I imagine the pre-war entries will be from where the day and month is recorded in the diary but not the year and it has accidentally been allocated to 1914. The Severn Bridge ones are near a place called Rogiet so it doesn't take a great leap of imagination to see how geo-tagging might get it wrong if it referred to a place that no longer exists today.

As you say, great animation and it does make it easier to find errors like these.

Glen

[Chris_Baker]

Trouble is, Glen, those things stand out because they are clearly wrong. It casts doubt over the rest of the data. We have seen from Ancestry's effort with the army service records the mess that can be made when paper records are transcribed and indexed by people who do not have background knowledge of the subject.

[spof]

I don't think Ancestry transcribed their service records. It looks more like an OCR job with a nod to making sure the names "sounded OK" (probably to a non-native English speaker though)

These diaries are tagged by, initially 5 but now 7, different people and the resulting tags are put through some type of analysis to get a best match. Not foolproof but probably as good as we'll get without a full transcription and that won't be allowed until NMP have recovered costs for digitising the diaries.

Overall, I agree with you though about accuracy concerns. Hopefully, the results will be seen in the "Lives" project soon but the other uses are still a bit vague and I'd like to see how it will be used before too long.