---
title: "package R : rwebstat"
author: 
- "Banque de France"
- "Jules Lecocq & Vincent Guegan"
date: "`r Sys.Date()`"
output:
  prettydoc::html_pretty:
    toc: true
    theme: architect
    highlight: github
    includes:
      in_header: header.html
vignette: >
  %\VignetteIndexEntry{rwebstat}
  %\VignetteEngine{knitr::knitr}
  %\VignetteEncoding{UTF-8}
  %\VignetteDepends{kableExtra,magrittr,htmltools}
---

```{r setup,echo=FALSE}
# setup chunk
NOT_CRAN <- identical(tolower(Sys.getenv("NOT_CRAN")),"true")
knitr::opts_chunk$set(purl = NOT_CRAN)
library(rwebstat)
```

```{r message=FALSE, warning=FALSE, include=FALSE}
library(kableExtra)
library(magrittr)
library(htmltools)
library(prettydoc)
```

```{r htmlTemplate, echo=FALSE}
if (require(htmltools)) {
img <- htmltools::img(src = knitr::image_uri("logo_webstat.png"),
               alt = 'logo',
               style = 'position:absolute; top:0; right:0; padding:10px;')

img_0 <- htmltools::img(src = knitr::image_uri("Nouveau_logo_Banque_de_France.jpg"),
               alt = 'logo',
               style = 'position:absolute; top:0; left:0; padding:10px',
               width = '180px',
               height = '85px;')

htmlhead <- paste0('
<script>
document.write(\'<div class="logos">',img,img_0,'</div>\')
</script>
')

readr::write_lines(htmlhead, path = "header.html") }
```


# Introduction

The [rwebstat](https://CRAN.R-project.org/package=rwebstat) package was created to facilitate access to the [webstat API](https://developer.webstat.banque-france.fr/). All the data are available on [Webstat](https://webstat.banque-france.fr/en/), the official external data provider website of [Banque de France](https://www.banque-france.fr/).

The first version was published on CRAN 2019-05-24.

## Requirement
The first step is to register on the API at  [link](https://developer.webstat.banque-france.fr/user/register). 

You can find operating procedure at these links ( [fr](https://webstat.banque-france.fr/Portail_API/Webstat_API_FR.pdf) and [en](https://webstat.banque-france.fr/Portail_API/Webstat_API_EN.pdf) )

Once done, you have to login and create an App which will give you an **API key** (personal Client ID). 

There are multiple ways to enter your **API key**. The simpler one is to store it in a global variable named "webstat_client_api" :
```{r, message=FALSE,warning=FALSE,eval=FALSE}
webstat_client_ID <- "3141592-65359-26535"
```

```{r message=FALSE, warning=FALSE, include=FALSE,eval=NOT_CRAN}
webstat_client_ID <- "300b933e-896f-4be2-b474-769894a56605"
```

```{r message=FALSE, warning=FALSE, include=FALSE,eval=NOT_CRAN}
proxy_bdf()
```

If you forget to create the variable, don't worry, the first function call will prompt you to enter it into RStudio Console.

### Proxy issues

<center><font color="red"> **Requirement for Banque of France employees**</font></center>

To use efficiently [rwebstat](https://CRAN.R-project.org/package=rwebstat) inside the Bank domain, you have to set your proxy with the <code>proxy_bdf()</code> function. Just enter your password when prompted.

```{r, message=FALSE,warning=FALSE,eval=FALSE}
proxy_bdf()
```

In any case, you need to set your proxy parameters (if you have any) in order to request the [Webstat API](https://developer.webstat.banque-france.fr/).

## Installation

You can easily install [rwebstat](https://CRAN.R-project.org/package=rwebstat) with the following code :

```{r, message=FALSE,warning=FALSE,eval=FALSE}
install.packages("rwebstat")
```

# Functionalities

This section will give you an overview of what you can do with [rwebstat](https://CRAN.R-project.org/package=rwebstat).

Data are stored in Series (time series). Series are stored in Datasets. 

Series id are Series keys ([sdmx](https://sdmx.org/) format). Datasets id are strings.

## Catalogues

We can easily recover Datasets and Series catalogues :

### Datasets

[Webstat](https://webstat.banque-france.fr/en/) offers more than 40 Datasets. The <code>w_datasets()</code> function returns the datasets catalogue :

```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
datasets <- w_datasets("en") # function call
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(datasets) <- NULL
datasets %>% 
        kable(row.names=NA) %>% 
        kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```

### Series

[Webstat](https://webstat.banque-france.fr/en/) corrently offers more than 40.000 Series. The <code>w_series_list()</code> function returns the series catalogue.

For example, we ask the **EXR** dataset catalogue (only top rows are displayed here) :

```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
EXR_series <- w_series_list("EXR") # function call
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(EXR_series) <- NULL
EXR_series %>% 
          head(5) %>%
          kable(row.names=NA) %>% 
          kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```

## Download data

Download all Series of a specific Dataset or an individual Serie with <code>w_data()</code> function :

```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
CPP_series_data <- w_data("CPP") # CPP is the smallest Dataset - 2 Series only
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(CPP_series_data) <- NULL
CPP_series_data %>% 
                head(10) %>%
                kable(row.names=NA) %>% 
                kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```

Download a specific Serie (<code>series_name</code> and <code>dataset_name</code> arguments are flexible) :

```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
USD_EUR <- w_data(dataset_name = "EXR", series_name = "M.USD.EUR.SP00.E") # exchange rate USD/EUR
USD_EUR <- w_data("EXR.M.USD.EUR.SP00.E")
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(USD_EUR) <- NULL
USD_EUR %>% 
        head(10) %>%
        kable(row.names=NA) %>% 
        kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```

## Search

We don't always know the exact Serie(s) key(s) we want to request. The <code>w_search()</code> function search <code>keyword</code> (regexp are accepted) inside catalogues.


### Datasets
For example, we look for the <code>keyword</code> "monetary" into the Dataset catalogue :

```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
s1 <- w_search(keyword="monetary",language="en")
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(s1) <- NULL
s1 %>% 
  head(5) %>%
  kable(row.names=NA) %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```


The <code>keyword</code> argument can be written in a regexp form to be more efficient.

```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
s2 <- w_search(keyword="\\wary",language="en") # use regexp to capture everything finising with "ary"
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(s2) <- NULL
s2 %>% 
  head(5) %>%
  kable(row.names=NA) %>% 
  kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```


### Series

We can pass all arguments from the <code>grep()</code> function family. If we don't want to search for a regexp expression, we pass the argument <code>fixed=TRUE</code>.

For example, we look for the exact word "dollar" into the EXR Series catalogue :
```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
s3 <- w_search("EXR",keyword="dollar",fixed=TRUE)
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(s3) <- NULL
s3 %>% 
   head(5) %>%
   kable(row.names=NA) %>% 
   kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```

## Metadata and structure

### Metadata

The <code>w_meta()</code> function return metadatas of a Serie. The language of the metadata will be the same as the language chosen for the Serie : 

```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
USD_EUR <- w_data("EXR.M.USD.EUR.SP00.E",language="fr")
USD_EUR_meta <- w_meta(USD_EUR)
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(USD_EUR_meta) <- NULL
USD_EUR_meta %>% 
                kable(row.names=NA) %>% 
                kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```

```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
USD_EUR <- w_data("EXR.M.USD.EUR.SP00.E",language="en")
USD_EUR_meta <- w_meta(USD_EUR)
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(USD_EUR_meta) <- NULL
USD_EUR_meta %>% 
                kable(row.names=NA) %>% 
                kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```

### Structure

The <code>w_structure()</code> function returns information on the structure of a specific Dataset as a R list :

```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
EXR_STRUCT <- w_structure("EXR",language="en")
class(EXR_STRUCT)
```

Elements of the structure list :
```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
names(EXR_STRUCT)
```

A Serie key ([SDMX](https://sdmx.org/) format) is a chain of strings separated with dots (M.USD.EUR.SP00.E). Each string is a dimension,

Dimensions of a Serie key from the EXR Dataset :

```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
EXR_STRUCT_dimensions <- EXR_STRUCT$keyFamily$dimensions[,1]
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(EXR_STRUCT_dimensions) <- NULL
EXR_STRUCT_dimensions %>% 
                      kable(row.names=NA) %>%
                      kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```

# Useful examples

We want to get back the last values of all Exchange rates Series (EXR Dataset) involving a "dollar" currency.

First we search the EXR Dataset for all the Series containing the "dollar" keyword :
```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
Series_dollar <- w_search("EXR",keyword="dollar",language="fr",fixed=TRUE)
dim(Series_dollar)
```

```{r include = FALSE}
len <- 24
```

```{r include = FALSE,eval=NOT_CRAN}
len <- dim(Series_dollar)[1]
```
We have a list of `r len` Series :
```{r, message=FALSE,warning=FALSE,eval=NOT_CRAN}
Series <- Series_dollar$SeriesKey
```

```{r echo=FALSE, message=FALSE, warning=FALSE,eval=NOT_CRAN}
rownames(Series) <- NULL
Series_p = data.frame(Series_dollar$SeriesKey,Series_dollar$Title)
Series_p %>% 
          kable(row.names=NA) %>%
          kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```

We then apply the <code>w_data()</code> function to the SeriesKey vector we found in the search :
```{r, message=FALSE,warning=FALSE,eval=FALSE}
Series_Data_list <-  lapply(Series,w_data)
```

# Support
Feel free to contact us with any question about the API or this package using this [e-mail address](mailto:gps.support@banque-france.fr?subject=[API][rwebstat]).