HDX Python Library Webinar [w/ subs]



00:00:01 – and to do is to hide the nuances of
00:00:03 – c-can from developers and made coding
00:00:06 – against HDX a much simpler endeavor
00:00:09 – so the idea is to use the library to
00:00:14 – push or pull data and 88 as simply as
00:00:18 – possible
00:00:20 – no there are a few ways in which is
00:00:23 – accomplished
00:00:25 – first of all the that the interface of
00:00:28 – the library is made of simple as
00:00:30 – possible
00:00:32 – the HDA I think somebody is not on mint
00:00:39 – could could you go on me
00:00:45 – sewing the HDX objects such as data sets
00:00:50 – and resources and gallery items are
00:00:53 – represented by python classes so when
00:00:57 – you look inside the HDX user interface
00:01:00 – and you see the various but the
00:01:05 – terminology that's being used in the HDX
00:01:07 – user interface then when you come
00:01:09 – through to use the HDX python library
00:01:11 – the terminology will be familiar to use
00:01:15 – and in this way we hope that this makes
00:01:17 – the learning curve vez of gentle as
00:01:20 – possible and enables you to get started
00:01:22 – with programming as quickly as possible
00:01:26 – so what with a please sort of design
00:01:35 – ideas behind the library i think the
00:01:38 – number one thing I would say about the
00:01:40 – whole library is to keep things as
00:01:41 – simple as possible so one way as i
00:01:45 – mentioned to do that was to avoid the
00:01:47 – sea can syntax so you don't need to know
00:01:52 – about the underlying platform of HDX
00:01:54 – when you want to look like a Python
00:01:58 – method then the class structure of the
00:02:02 – library was designed to be as logical as
00:02:06 – possible so HDX a dataset can contain
00:02:10 – zero or more resources for those not too
00:02:15 – familiar with ATX basically what you
00:02:20 – have a top level of the organization
00:02:23 – many of you will be apart so the
00:02:28 – organization could be the red cloth then
00:02:30 – the lake las can have multiple datasets
00:02:34 – within HDX and each data set can have a
00:02:40 – little more resources on these resources
00:02:42 – are basically files the files can be
00:02:46 – hosted on HDX or they can be hosted
00:02:52 – externally on service or Google dogs or
00:02:56 – other other places
00:03:00 – the other thing that HDX datasets have
00:03:03 – our gallery items
00:03:09 – so in the diagram that you hopefully can
00:03:13 – see in front of you there's a it
00:03:16 – basically shows the class structure of
00:03:19 – the library and I think you can see that
00:03:23 – it's it's really quite simple
00:03:27 – we get it maybe you can make it a little
00:03:28 – bigger
00:03:31 – ok
00:03:33 – well yeah that's great
00:03:39 – so this is basically a UML diagram that
00:03:45 – shows you how how the structure of the
00:03:48 – library and the classes in the library
00:03:51 – is is and
00:03:55 – [Music]
00:03:58 – the other important
00:03:59 – aspect is how to actually change
00:04:02 – information within the datasets I wanted
00:04:06 – that to be as simple as possible as well
00:04:09 – so and each of the upper classes
00:04:13 – basically allows you to use addiction me
00:04:18 – method which is basically just square
00:04:20 – brackets and Python to to handle the
00:04:23 – metadata so for example if you want to
00:04:26 – set for the name of the data sent as you
00:04:29 – can see in number three it's really just
00:04:32 – a case for putting name in square
00:04:35 – brackets and then equals to whatever the
00:04:37 – value you wanted to be but where things
00:04:41 – are a bit more complicated behind the
00:04:44 – things then there are some helper
00:04:46 – methods so for example with dealing with
00:04:50 – dates you might want to feed in
00:04:53 – different date formats and and the HDX
00:04:56 – library has a method to help you with
00:05:00 – that which I'll come to later
00:05:04 – often with data sets a lot of the beta
00:05:07 – but the metadata is static so it doesn't
00:05:12 – change over time and for this leaving
00:05:16 – it's helpful to be able to just import
00:05:19 – all the platform that from a file so the
00:05:22 – library allows you to import from either
00:05:24 – i am or fall or adjacent farm animal
00:05:29 – being recommended because it's very easy
00:05:32 – for for humans to meet
00:05:38 – and in addition letter from utility
00:05:40 – functions within the library which
00:05:42 – confined within the documentation and
00:05:47 – leave may be helpful for for for various
00:05:49 – stuff that you might need to do
00:05:55 – a thing as a program that I've often
00:05:57 – found is that the setup required to get
00:06:02 – going with any new library can often be
00:06:06 – the biggest challenge and some things
00:06:10 – are often when when starting a new
00:06:13 – project
00:06:14 – left left behind or forgotten about one
00:06:18 – of those things is logging so the light
00:06:21 – we provides logging facilities so that
00:06:25 – when debugging you don't have print
00:06:28 – statements all over the place
00:06:31 – and by default it sets up logging so a
00:06:35 – lot a lot of it is hidden away from the
00:06:38 – user so for those of you who answer the
00:06:42 – poll that you are beginners this will
00:06:45 – definitely make your life easier
00:06:49 – and similarly there's the possibility if
00:06:53 – you're writing something that pushes
00:06:56 – data into HDX and you want to be alerted
00:06:58 – to failures by email at that facility is
00:07:03 – also available in the library
00:07:08 – the aspect is a configuration so that's
00:07:12 – made that simple of possible by a
00:07:14 – configuration class and i will show
00:07:17 – later healthy you sense
00:07:22 – I'm if the library fails for some reason
00:07:27 – then the logging is used to make it
00:07:30 – clear what operation was being performed
00:07:32 – at a point of failure and the idea is to
00:07:40 – make it as obvious as possible
00:07:43 – what's gone wrong so exceptions are
00:07:45 – flown rather than just returning a false
00:07:48 – alarm none and so that it's clear what
00:07:53 – needs to be fixed or changed
00:07:57 – of you who are beginners again or even
00:08:00 – experienced programmers there are what
00:08:04 – are called fast-forward be for basically
00:08:07 – it means with simplifying the setup as
00:08:11 – much as possible so i will have to show
00:08:15 – these little another important aspect
00:08:19 – from of any libraries its documentation
00:08:23 – and for the documentation is extremely
00:08:27 – comprehensive service lively as we
00:08:31 – method has has documentation
00:08:38 – and they're one of the helpful things
00:08:41 – that your development environment
00:08:43 – example if you use PI term can take
00:08:46 – advantage of the provided documentation
00:08:48 – one of the ways in which it can do that
00:08:52 – is through Titans so Titan is that we
00:08:59 – normally have types only one there is
00:09:02 – not a strongly typed language but with
00:09:07 – the type hints these can help you can
00:09:10 – make sure you don't make mistakes when
00:09:14 – you're calling functions and any of
00:09:17 – pulling the methods of the classes and
00:09:18 – you know I'll what types of expected for
00:09:21 – each parameter of the method
00:09:26 – other way of simplifying things he says
00:09:29 – a lot of default parameters so if you
00:09:33 – don't specify things a lot of things
00:09:35 – will if you don't specify plant with a
00:09:39 – lot of them wore already have the faults
00:09:41 – which you can use
00:09:47 – at this point i will switch to the
00:09:51 – demonstration to show how easy it is to
00:09:55 – get going with the local before you do
00:09:58 – that there was one question from andy
00:10:00 – smith for those of us who are familiar
00:10:03 – with you can is there somewhere where
00:10:05 – it's possible to look up the mapping of
00:10:07 – pecan HDX terms you can to HDX terms you
00:10:12 – know that's a very good question
00:10:13 – actually the the best way is to come off
00:10:19 – me I but now you've mentioned with Andy
00:10:22 – I think it would be a good idea for us
00:10:25 – to document this and to give you some
00:10:29 – idea i think Pete we can as a dataset
00:10:33 – although they used to call them packages
00:10:34 – resources of the same though that sort
00:10:39 – of trickiest one if you like her gallery
00:10:41 – items which map to see can related items
00:10:45 – underneath but anyway if you need more
00:10:47 – information please feel free to contact
00:10:50 – me about it
00:10:51 – thanks thanks
00:11:01 – ok so I'm going
00:11:04 – to show you how to get going with the
00:11:08 – library
00:11:09 – I mean I think he's going to one through
00:11:12 – the tutorial that you can find on the
00:11:14 – library's get up page so you can easily
00:11:17 – follow through the steps but that I am
00:11:20 – about to show you in your own time after
00:11:23 – lips
00:11:27 – I would highlight is that the library is
00:11:30 – currently python3 only and that's
00:11:34 – because of the tight hints that i
00:11:37 – mentioned previously that of a feature
00:11:38 – of Python 3
00:11:40 – um but I've had one request so far for
00:11:44 – python to support and if there is
00:11:46 – sufficient demand for this then I'm
00:11:49 – waiting
00:11:50 – porting the library 2.2 is is certainly
00:11:53 – something that we will consider
00:12:02 – I'm going to
00:12:04 – not show like the full all of the stages
00:12:09 – of setting up python and faithful that
00:12:12 – open installing the virtual environment
00:12:15 – and if you can do in your own time
00:12:20 – um but what I will show if from step 3
00:12:24 – which is human you've got that working
00:12:28 – python installation with virtual and
00:12:30 – installed how to then use that to to use
00:12:35 – the library
00:12:49 – so the first step is to set
00:12:52 – an environment for those unfamiliar with
00:12:55 – what that is it's basically a sandbox
00:12:59 – that you can play around them so that if
00:13:03 – you mess up the installation in the it
00:13:06 – doesn't really matter so for those of
00:13:07 – you who are beginners this is a practice
00:13:12 – using virtual environment of a practice
00:13:16 – that I would like a mint
00:13:19 – the next step is we activate the virtual
00:13:23 – environment
00:13:25 – then we install the library now
00:13:36 – of the diversion to a actual version of
00:13:40 – the library so we're in advance at
00:13:43 – version here it needs to be altered to
00:13:46 – promote the latest version of the
00:13:49 – library available and that you can find
00:13:53 – from the link shown on the tutorial page
00:13:59 – so if you click on that link
00:14:06 – you can see the different versions of
00:14:08 – the library and we can see that the
00:14:10 – current version this is pointed 1b0 on
00:14:14 – 81
00:14:20 – we change village
00:14:22 – no point eight one
00:14:25 – and then we wait
00:14:26 – [Music]
00:14:30 – library and all of its dependencies
00:14:36 – I
00:14:38 – get any errors in this process this wall
00:14:41 – that almost let me be with one of the
00:14:45 – dependences with the library
00:14:47 – um and if you look at step 5 which i
00:14:50 – won't go into now that's one of the
00:14:54 – usual reasons for failure
00:14:56 – so you can you can see that in the
00:14:57 – tutorial
00:15:00 – now with
00:15:02 – all set up with the library so we can
00:15:05 – launch python
00:15:12 – so the first step is
00:15:14 – you
00:15:15 – port some requirement classes
00:15:21 – so we need
00:15:23 – regulation and the data set because he's
00:15:26 – the things that we want to to work with
00:15:28 – now
00:15:32 – when you create the configuration you
00:15:36 – have the option to use HDX meet only
00:15:40 – which if you're a beginner
00:15:43 – perhaps that's a good way to start so
00:15:45 – that you don't have to chance of
00:15:47 – accidentally writing something um
00:15:50 – however we we do have a test websites
00:15:54 – allow organizations to play with which
00:15:58 – we call features so it wouldn't be a
00:16:00 – disaster if you look for that one
00:16:03 – so in the configuration command
00:16:10 – patrol paste here
00:16:14 – you can
00:16:16 – TX site which is listed as feature and
00:16:19 – HDX read-only is a flag that went to
00:16:22 – true in this case and this woman you can
00:16:25 – only read from HDX the other thing that
00:16:28 – means is that you don't need a special
00:16:30 – api key because in order to light the
00:16:33 – HDX you need to register on the website
00:16:36 – and obtain a special key and called an
00:16:41 – API key and and set that
00:16:47 – blue if you do wish to write the HDX and
00:16:49 – you wish to set up key then there's a
00:16:53 – number of steps will actually listed in
00:16:56 – the tutorial at the top on getting
00:16:58 – started about creating the API key file
00:17:01 – and it in the seven steps listed it
00:17:05 – tells you blow-by-blow exactly where to
00:17:08 – find the API key and what the do put
00:17:11 – eventually you create the file called
00:17:14 – dot HDX key in your own directory and
00:17:19 – paste values that you find on earth on
00:17:23 – the HDX website influence
00:17:30 – but I have a key so I'm going and I
00:17:34 – intent of light as well so i'm going to
00:17:36 – move the HDX we'd only flag
00:17:43 – I'm another observation would make about
00:17:46 – this is this configuration setup has
00:17:49 – changed since the last time I i did it
00:17:52 – did a webinar about this likely before
00:17:55 – configuration objects needed to be
00:17:57 – passed to thank the sets and resources
00:18:00 – and so forth
00:18:02 – now you need only our like this one
00:18:05 – command to set up the configuration
00:18:08 – globally and and this makes life a lot
00:18:11 – easier because it's one less piece of
00:18:14 – information that needs to be passed
00:18:15 – around
00:18:20 – that we're interested in is from an
00:18:23 – organisation called a clad and its
00:18:26 – conflict better for Africa um it's this
00:18:30 – one
00:18:31 – um and what we
00:18:35 – going to do is we're going through load
00:18:38 – this data set using the HDX python
00:18:41 – library then change this state of data
00:18:44 – set safe the data center and then we'll
00:18:49 – have a look to to make sure it's
00:18:51 – actually changed
00:19:00 – so reading from
00:19:04 – alright
00:19:05 – very simple process it's just dataset
00:19:10 – don't be too nice
00:19:11 – tanks and the name of the the data set
00:19:15 – that you want I'll I'll come later to
00:19:17 – help to actually search for data sets
00:19:19 – but for now we'll assume that you have
00:19:21 – the name of the data set
00:19:27 – well what the date of the data set
00:19:32 – here I'm using one of the benefits
00:19:37 – helping that some of the date is printed
00:19:40 – in a in a readable format and you can
00:19:44 – actually specify which format you would
00:19:46 – like that the data being if you wish
00:19:52 – changes
00:19:56 – he liked it too HDX
00:20:00 – this last step so we set the data set
00:20:04 – date with the format that were using and
00:20:07 – then we call update in htx and at this
00:20:10 – point communication will occur with the
00:20:13 – HDX website
00:20:23 – so if we go to
00:20:26 – to this date event that we've updated
00:20:29 – and now i refresh the page
00:20:36 – of the state of the
00:20:37 – the set has changed to to what I had a
00:20:41 – secretary I the 26 of july 2015
00:20:52 – we'll just change it back
00:21:06 – [Music]
00:21:21 – and here you can see that the people
00:21:25 – that data in change back
00:21:36 – Oh
00:21:38 – one of the things you might need to do
00:21:40 – is search on a gtx 480 service and you
00:21:44 – can do that using the user interface of
00:21:47 – course on the website but it's also
00:21:49 – possible to do that you can HDX python
00:21:52 – library so you have an example of
00:21:55 – searching for data sets with acted in
00:22:00 – the main water elsewhere in the metadata
00:22:03 – and we want to just return the first 10
00:22:07 – results in this case
00:22:14 – a lot of information return this is all
00:22:19 – of the metadata for 410 datasets
00:22:22 – and if we want to within the data set
00:22:27 – there are there were multiple with one
00:22:31 – all the more resources and so if we want
00:22:37 – to get all of the resources from these
00:22:39 – datasets in in one go
00:22:41 – I'm we can use this helper method called
00:22:46 – get all these sources which flattens are
00:22:49 – and the list of data sets and the terms
00:22:52 – of all of the resources we would want to
00:22:56 – do this because we might want to get to
00:22:58 – the the files and download them so if we
00:23:04 – run this command
00:23:12 – and then have a look at the output
00:23:15 – might you're breaking up a little bit
00:23:18 – i'm not quite sure what happened but
00:23:20 – maybe you can repealing innocent
00:23:23 – okay um we can can you hear me now
00:23:29 – yeah yeah okay we can drink the
00:23:33 – resources that we just pulled from the
00:23:36 – the 10 datasets which we found in our
00:23:39 – search and this will give you an idea of
00:23:42 – what the legal structure looks like them
00:23:48 – but say we are interested in the the
00:23:51 – first of these sources and and we want
00:23:53 – to download it we can simply download
00:24:00 – and run at the first object that the
00:24:04 – file of that's in the first reasons by
00:24:08 – using the following commands
00:24:15 – be in this case i haven't specified a a
00:24:18 – path you are able to specify the part of
00:24:21 – you if you wish but we'll see where it's
00:24:25 – downloaded we can see it's been
00:24:27 – downloaded to tent if we're looking
00:24:33 – and he should be able to fire that we
00:24:37 – are so we've downloaded a unactivated
00:24:42 – legals which in this case with a excel
00:24:46 – spreadsheet
00:24:50 – you're still breaking up i'm not sure if
00:24:53 – it's just me your other son I am host so
00:24:58 – I'm hoping it's not going to affect what
00:25:00 – are they doing ok how do you how do it
00:25:05 – every day people other people think they
00:25:06 – can hear it so all right
00:25:08 – ok ok ok so once you've finished with
00:25:17 – your virtual environment you can remove
00:25:23 – it and deleted on so this shows that
00:25:30 – it's completely self-contained and you
00:25:32 – don't have to worry about what you've
00:25:36 – done to your Python set up within that
00:25:38 – virgin employment
00:25:50 – so I'm
00:25:54 – talk a little bit about
00:25:58 – from what
00:26:00 – other features of the library which you
00:26:02 – can find this for random order you to
00:26:07 – that I wanted to check in with everyone
00:26:09 – and see if anyone had questions you can
00:26:13 – go ahead and type in the chat or just
00:26:15 – unmute and an answer your question
00:26:26 – that sister
00:26:28 – I have a quick question I I don't know
00:26:32 – if you plan to do that there are coming
00:26:35 – great presentation but it's do you have
00:26:37 – an example of use case because here we
00:26:40 – can see security or so
00:26:43 – api all of us to manage or data but is
00:26:48 – idea Eddie example existing from
00:26:52 – different organizations of different
00:26:55 – setup just be interesting for social
00:26:59 – needs / yes that's a good question and
00:27:04 – the answer is that I'm going to come to
00:27:07 – that there is indeed of such an example
00:27:10 – so i will i will show that actually
00:27:15 – all the any other yes there's there's
00:27:18 – one more question someone is the nice
00:27:21 – library how long did it take to build
00:27:23 – and Dale I that's a good question from i
00:27:30 – started this around july but this has
00:27:33 – not been a full-time piece of work there
00:27:36 – are many other pieces of work as well
00:27:37 – but it but it was initiated in July the
00:27:42 – eastern has been something that I've
00:27:44 – worked on about time since then it's
00:27:47 – been through a number of releases and
00:27:50 – the partner organizations who have
00:27:53 – started to use the library have been
00:27:55 – very helpful in giving feedback and help
00:27:58 – me to CEO how to make improvements to
00:28:01 – the library
00:28:02 – great so Dale is actually from the
00:28:06 – international aid transparency
00:28:07 – initiative erp and they are doing a
00:28:10 – similar library so he want to keep in
00:28:13 – touch and then we would happen to us
00:28:16 – three different and that andy says he
00:28:19 – has a use case is required late
00:28:23 – ok i'm very happy to the event ok
00:28:26 – alright so let's keep on how am i doing
00:28:31 – for time because so it is 1010 there's
00:28:34 – 20 more minutes okay i'll find the loss
00:28:37 – but reason because i think people can
00:28:39 – read through a lot of this in their in
00:28:41 – their own time I'll just highlight a few
00:28:45 – features of the library and i had
00:28:48 – mentioned before that you can use
00:28:51 – something called a facade and that is
00:28:56 – designed to make the library a bit
00:28:58 – easier to use
00:29:00 – I mean I think the best way i can show
00:29:03 – that it's with the example that comes
00:29:05 – later so I'll all come back for that one
00:29:11 – configuration but the parameters that
00:29:15 – you can change all listed in the readme
00:29:18 – in the github
00:29:22 – and similarly for the logging there's
00:29:24 – also parameters that that eventually
00:29:28 – specify where the configuration
00:29:30 – information is coming from and what the
00:29:32 – default is if you don't specify it
00:29:40 – more interesting
00:29:42 – the
00:29:43 – patients that are available on on HDX
00:29:45 – objects so aside from meeting through
00:29:49 – all the api docs oh that was there were
00:29:52 – fun highlighted here and so obviously
00:29:55 – you you need to be able to read from HDX
00:29:59 – and you can read data sets or legal
00:30:01 – there's or gallery items you can search
00:30:05 – using the search in htx method that we
00:30:09 – looked at earlier
00:30:12 – on you can create that effect from
00:30:15 – scratch of course and that can be done
00:30:18 – either from a file
00:30:21 – yeah more follower Jason filed or or
00:30:24 – simply from the dictionary passed into
00:30:26 – the data set
00:30:35 – if we
00:30:36 – [Music]
00:30:37 – look at it from the accurate normal fall
00:30:40 – then you can see some of the fields that
00:30:45 – can be filled in and I can point you to
00:30:51 – that example
00:30:55 – once you put all of the information you
00:30:59 – want into your dataset object you just
00:31:03 – call creating HDX to have it be created
00:31:06 – on the HDX website or if you have loaded
00:31:10 – an existing one and you're making some
00:31:13 – changes you call updated HDX you also
00:31:17 – have the option to delete as well
00:31:30 – on a dataset
00:31:32 – on else was nature's you can have within
00:31:34 – a dataset you can have one or more
00:31:36 – resources or one more gallery items of
00:31:40 – the divil my apologies and for that you
00:31:43 – to add them to the basic you use this
00:31:47 – method add update resources or add
00:31:50 – update gallery gallery and then you pass
00:31:55 – in a list of resources that you all
00:31:57 – gallery items that you wish to to add
00:32:00 – and similarly you can get all the
00:32:03 – resources former data set with the
00:32:05 – getresources metal
00:32:10 – and if that's what
00:32:13 – and in the tutorial if you have a number
00:32:15 – of datasets and you wish to get all of
00:32:17 – the resources out of those datasets in
00:32:20 – one go you can call the gate all
00:32:23 – resources method on the dataset class
00:32:26 – I'm weathered one of the areas where the
00:32:31 – library has undergone a number of his
00:32:33 – improvements in what methods to help
00:32:37 – users with a certain field sort of a
00:32:40 – little more tricky to set up sometimes
00:32:42 – because of the way they can implemented
00:32:44 – within secant so you have getters and
00:32:49 – setters for the data set date where you
00:32:52 – can specify the format of the bait that
00:32:56 – you would like for the expected update
00:32:59 – frequency field which by the way is now
00:33:02 – a mandatory field it's no longer
00:33:04 – something that's optional you have to
00:33:08 – fit in on the HDX website or through the
00:33:12 – API um you have a number of firsts
00:33:16 – frequencies so if you look in the HDX
00:33:19 – web interface you'll see every day every
00:33:21 – week and so forth
00:33:22 – um so the API has a setter that allows
00:33:27 – you to use the same values that you see
00:33:31 – on the HDX website like everyday left
00:33:34 – every week lot of them popping in a
00:33:36 – number
00:33:39 – uh
00:33:40 – only you can add load locations on the
00:33:43 – HDX website so the light be provides a
00:33:47 – means of adding to a dataset country
00:33:51 – locations as I saw three country codes
00:33:54 – or if you wish to add all the countries
00:33:58 – in a confident you can add a compliment
00:34:02 – using a two-letter code and the library
00:34:04 – is a bit flexible there because you can
00:34:06 – put in an approximate name of search and
00:34:10 – it will attempt to guess so if you put
00:34:12 – Africa or something like that and try to
00:34:15 – work out which continent so that you're
00:34:20 – talking about so you don't have to
00:34:22 – necessarily put in the code although
00:34:24 – it's recommended to be absolutely clear
00:34:26 – and what what is being put in
00:34:31 – and they don't they will also help us
00:34:33 – for adding tags field data
00:34:39 – um
00:34:40 – he on resources you have the facility to
00:34:43 – download them and also if you wish to
00:34:47 – store your wheels within HDX lot of the
00:34:50 – note on your own server then have the
00:34:54 – option to specify the path to a final
00:34:57 – that you wish to upload to HDX
00:35:04 – within HDX
00:35:05 – you have that is the data preview
00:35:07 – feature and I i think i will leave that
00:35:12 – for another time but essentially there
00:35:14 – are methods to to deal with that
00:35:17 – so now we come to a working example on
00:35:21 – so if we want to create an example to
00:35:24 – ranch we can use this fold
00:35:32 – facility that i mentioned before and as
00:35:36 – you can see the sort of it provides the
00:35:39 – template really so you you would just
00:35:43 – fill in your code within the main method
00:35:45 – and all the third logging and
00:35:49 – configuration those all set up behind
00:35:52 – the things so
00:35:57 – if you in this case provide a and
00:36:00 – generate data set function somewhere
00:36:03 – other than becomes a dataset then this
00:36:08 – will be enough to get you going with
00:36:11 – creating data sense
00:36:14 – so
00:36:16 – for a fuller example of that you can go
00:36:19 – to get up to the accurate example and
00:36:24 – have a look at the London py and
00:36:26 – accurate africa PTY and the conflict
00:36:28 – folder will see the static yeah more
00:36:33 – file at the static metadata within a
00:36:36 – more fall and also all the other and
00:36:39 – configuration and the setup that's done
00:36:43 – in order to light datasets into the
00:36:45 – accolade organization and a cloud
00:36:52 – actually was one of the ways in which
00:36:53 – and the the design of the library was in
00:36:57 – truth in the process of writing the code
00:36:59 – for putting information into activate it
00:37:02 – became clear where there were
00:37:05 – improvements that could be made
00:37:10 – so I encourage you to try the library
00:37:14 – and to file requests for new features to
00:37:18 – report bugs and even better
00:37:23 – please feel free to contribute to the
00:37:25 – development of the library would be very
00:37:27 – happy to have volunteers to work on the
00:37:31 – library and make improvements that can
00:37:34 – but the library is entirely public love
00:37:37 – coded open and it's on github so please
00:37:42 – feel free to to make pull request to add
00:37:45 – features or if you're not a programmer
00:37:49 – that you need a feature her feel free to
00:37:52 – to request my features directly with us
00:37:54 – and also help us to identify at places
00:37:59 – where the library could be used then
00:38:01 – it'd be really grateful for that
00:38:05 – so the point yeah yes we can get for
00:38:09 – question 4 think yeah there's just one
00:38:12 – from andy again about sound so he's
00:38:17 – saying that HDX have stricter holds
00:38:19 – about metadata and plain c-can is there
00:38:22 – a way to validate the meditative luckily
00:38:24 – with the library or is it only valid
00:38:26 – when you attempt related back to HBS how
00:38:30 – that's a good question
00:38:32 – yes there is a way to validate before
00:38:37 – lighting and just trying to see if I've
00:38:39 – is so we have a only on paper sets or
00:38:44 – resources you have a medical check
00:38:47 – required fields and you can call that
00:38:51 – method and it will tell you whether
00:38:53 – there are any missing field so that's
00:38:58 – something we have we have thoughtful and
00:39:01 – that method is obviously used by the
00:39:02 – library itself for its own internal
00:39:04 – checking
00:39:05 – ok girls asking does HDX prevent
00:39:11 – publication of invalid that uh no you
00:39:16 – know what i mean if you put inside a
00:39:20 – fierce Felix else um
00:39:22 – and later that's invalid in some way
00:39:25 – then the spreadsheet will still be made
00:39:29 – available on ATX so but isn't that means
00:39:34 – by which you can validate the data that
00:39:36 – that's in your measles
00:39:39 – although that is something that the HDX
00:39:41 – team is considering i'm particularly in
00:39:45 – the guards for using the hexyl language
00:39:51 – with pixel is with with X with you if
00:39:57 – you add to your spreadsheet ocsc a love
00:40:01 – that specialize in a standardized way
00:40:04 – what turn the columns are and so we hope
00:40:09 – to build on this standardized language
00:40:12 – and music such as checking and
00:40:15 – validating data in future
00:40:17 – ok are the car lab team is asking is
00:40:23 – there a way to automatically search and
00:40:25 – download our resource from a particular
00:40:27 – data set using the library
00:40:31 – it depends what you mean by automatic I
00:40:35 – mean there's certainly search facilities
00:40:37 – that you can use so you can you just
00:40:41 – specify have searched term base you can
00:40:47 – specify and filters and that search
00:40:51 – functionality of actually within the fee
00:40:56 – can rest api and the the library just
00:41:01 – exposes that at the moment it doesn't
00:41:02 – add add to that functionality so
00:41:06 – [Music]
00:41:08 – so yes you can find out in the
00:41:10 – vulcanization okay i mean the car team
00:41:15 – let me know if that answers your
00:41:16 – question
00:41:17 – otherwise we can just follow up
00:41:18 – Sebastian is asking does htx related
00:41:22 – somehow to net CDF I let me google that
00:41:30 – feed ya when i get anything done you and
00:41:34 – answer your question on that Network
00:41:38 – common three former probably know it
00:41:43 – doesn't relate to that but it's
00:41:45 – interesting that you leave her a little
00:41:48 – restaurant the other so yeah okay yeah
00:41:52 – so uh yeah that's what the essay is a
00:41:57 – data management warm of that was a
00:42:03 – developer some of ages ago a the for
00:42:10 – NASA I think okay we just actually very
00:42:15 – very much used is especially with the
00:42:18 – climate data and it there so after
00:42:24 – seeing the example shown here uh I
00:42:29 – noticed a few similarities so I thought
00:42:31 – that there may have been some form of
00:42:35 – integration between formats but
00:42:36 – certainly it's nothing it's not the case
00:42:40 – no not not at the moment I'm ready is
00:42:43 – actually the first of my mind I've heard
00:42:45 – of this format but um we will certainly
00:42:47 – have a look at it and something there
00:42:50 – for all of that
00:42:52 – at least in the very scientific
00:42:58 – situations and environments is a very
00:43:05 – used and unfortunately kind of amenities
00:43:11 – can get messy when trying to to use have
00:43:16 – the install the door the relative a
00:43:19 – library for python are other languages
00:43:24 – so well anyway it's not there's
00:43:28 – something to look into it and you thank
00:43:31 – you for raising it with us
00:43:33 – thank you baby is asking another
00:43:35 – question he saying I would be interested
00:43:37 – to push data HDX and not to get it we
00:43:40 – can update the metadata but could be
00:43:42 – also pushed or send data to HDX as well
00:43:45 – I would be interested to automate
00:43:47 – automatized regular updates of smile for
00:43:51 – Christmas replacing existing file and
00:43:53 – upload date
00:43:55 – yes that's a good question and and the
00:43:58 – simple answer to the question is yes you
00:44:00 – can push data to a gtx and you have the
00:44:05 – facility to upload into the ATX foil
00:44:07 – with all the father that you want them
00:44:09 – and if you keep the resource ID the same
00:44:14 – from one upload to the next then then it
00:44:19 – will be a direct replacement for the
00:44:22 – fall so that the URL that you use to
00:44:25 – access the fall will stay the same so
00:44:28 – you can definitely do now what is that
00:44:31 – you requested okay and that sort of
00:44:35 – float for those who attended the
00:44:37 – previous webinar that's actually a new
00:44:39 – feature of the library since then
00:44:43 – ok adele is asking a remedy said thank
00:44:50 – you I there was asking if it's all the
00:44:54 – docs for the library are in the github
00:44:56 – readme files and it's not good
00:44:58 – could you general that quickly now ok so
00:45:02 – we have some a few places one is on our
00:45:07 – public consultations for the design
00:45:10 – philosophy here for lighting um so that
00:45:14 – one can have you already tragedies
00:45:17 – thinks that you're sure yeah
00:45:19 – yes i did there there above image at my
00:45:23 – wedding though so there's the confluence
00:45:28 – light which is sort of a top-level view
00:45:31 – of the library and then there's the
00:45:33 – getup we'd me which contains the
00:45:35 – tutorial and then on top of that you
00:45:39 – have because the link actually from the
00:45:42 – github page to the api docs strong point
00:45:48 – on I was actually a link to their
00:45:51 – compliments page from the get-go page
00:45:53 – once you have that you can get there and
00:45:56 – then the API documentation is also
00:46:00 – listed on that use you can see launch
00:46:05 – this you can see all of the methods are
00:46:11 – available on a date alert for example
00:46:14 – here home so that's all that the
00:46:18 – documentation available at this time
00:46:22 – okay i think that is it for the
00:46:28 – questions in the chat and you went out
00:46:31 – yes Chris happy that is asking if you
00:46:34 – can give us a sneak peek if HDX plans on
00:46:38 – linking with mobile data collection
00:46:40 – platforms to publish data great question
00:46:43 – yes that's definitely a future plan by
00:46:48 – mobile platforms
00:46:50 – well the one we're looking at the moment
00:46:53 – is a kobo so that's what project that
00:46:58 – i'm on the way at the moment and we're
00:47:01 – definitely looking at this
00:47:02 – yeah and in fact we have we've just
00:47:06 – completed some user research in west
00:47:08 – africa and we'll be sharing more about
00:47:10 – that on the blog
00:47:14 – any other questions we've got five more
00:47:20 – minutes up we don't have more minutes
00:47:22 – they have a couple more minutes
00:47:24 – one last thing from Chris you want to
00:47:28 – just unmute Chris
00:47:37 – he's asking about data visualization and
00:47:40 – power bi but you know I was a great
00:47:46 – position either kill you don't think
00:47:49 – about what I just wanted to ask you i
00:47:53 – don't think i can find you searching how
00:47:54 – to make HDX were did a smart data
00:47:57 – analytics and power bi specifically
00:48:00 – that's not something with we looking at
00:48:06 – the moment I mean power bi is one of the
00:48:08 – number of this kind of platform so we
00:48:11 – would need to sort of know how many
00:48:14 – people are interested in and using that
00:48:18 – platform specifically if we decide to go
00:48:20 – ahead and and support it and I know that
00:48:25 – there's Google is working on this kind
00:48:28 – of platform and the others as well so
00:48:30 – it's a question of with we have limited
00:48:32 – resources so we have to be careful about
00:48:35 – which ones we we target but if some that
00:48:39 – one of the things i call again for
00:48:41 – volunteer that there's any volunteers
00:48:43 – who wants to look at the kind of project
00:48:45 – and we we would be very happy to be as
00:48:47 – supportive as we can in this kind of
00:48:49 – work with
00:48:51 – well I mean that I'm sorry turn this
00:48:53 – into a discussion just gonna be very
00:48:55 – soon my reply i work for I'm and we're
00:48:58 – trying to convert some weight gather
00:49:00 – more humanitarian big data and I think
00:49:02 – HDX is a very crucial component in this
00:49:04 – and this project so what we're looking
00:49:07 – at is basically to have some people
00:49:09 – don't know what they need but if we
00:49:10 – guide them properly think that will
00:49:13 – reach a sort of like a productive
00:49:14 – productive outcome and the thing is that
00:49:17 – lots of data being generated not really
00:49:20 – share the HDX we trying to bridge the
00:49:22 – gap do that people share the data and
00:49:25 – that and at the same time try to
00:49:28 – understand the benefits of linking to
00:49:31 – such a platform linking to multiple
00:49:33 – sources using powerful smart and other
00:49:35 – six programs and I'm making up
00:49:36 – identifying patterns trends improving
00:49:39 – the responses cetera
00:49:40 – sorry for taking too long thank you so
00:49:42 – much no thank you very much for joining
00:49:45 – us about your look
00:49:46 – yeah yeah no problem
00:49:49 – so Mike if you just want to give me back
00:49:51 – the screen I will pull up the final
00:49:54 – slide
00:49:55 – can I take it back myself yes here we go
00:50:00 – ok so can you see my final here this is
00:50:07 – all the contact information i'm not
00:50:15 – doing it i'm looking okay
00:50:24 – ok
00:50:30 – okay so yeah here is mike's email thank
00:50:33 – you so much for joining us
00:50:35 – i sent a link in the chat to the
00:50:37 – feedback form we'd love to hear from you
00:50:39 – and at my says we're looking for
00:50:43 – volunteers that would be great to just
00:50:47 – keep in pops we're on twitter where
00:50:49 – we're always happy to hear from you by
00:50:53 – whatever means possible and and we look
00:50:57 – forward to keeping the conversation
00:50:58 – going
00:50:59 – so thanks again for for everyone joining
00:51:03 – us and helps here for me against thank
00:51:08 – you for attending and thank you for this
00:51:09 – week
00:51:12 – alright thank you bye


Video Url:
http://youtu.be/I62ITL7mt5s

Comments

comments