Monitoring is a crucial step for the implementation of RIS3 projects. It usually encompasses all sorts of activities that have to do with the collection and processing of information about the achievement of expected results and the degree of implementation of policy measures (Gianelle and Kleibrink 2015). In general, monitoring can be effectively interpreted in combination with the general RIS3 context, due to its strong connection with the RIS3 strategy structure. This means that depending on the diversified characteristics of the various RIS3 strategies that are implemented in EU countries, monitoring can take different forms. Furthermore, given the fact that monitoring can be received as an information gathering mechanism, one of its main utilities is to constitute a base upon which a periodic refinement and refocusing could be made.

Monitoring is achieved apart from interactive dashboards, through a meaningful report Monitoring Report; static spreadsheets and visualisations (Figure 1). With a few clicks, the application permit users generate summaries of any data sets, enabling them to easily make comparisons and write a summary. The contents of the report are customizable and adjusted to each region and the final report can be downloaded as a PDF or Word file.

RIS3 Monitoring key blocks

Figure 1. RIS3 Monitoring key blocks

One of the core elements for the application 6.1 is the visualisation process. Visualisations make it easier to understand information and detect key issues that otherwise could be lost in a spreadsheet. For this reason, many different approaches for visualisation are adopted in the application including box plots, bar charts and tables etc. Figure 1 provides the rationale behind this ONLINE S3’s application. There are six (6) discrete stages in the "RIS3 Monitoring" application.

As it is shown in Figure 2 the core block is the Monitoring Report which the user can elaborate using data from the blocks: Output indicators, Budget data, User satisfaction and Balance Scorecard. The monitoring report is structured to present in detail: 1) The objectives of the investment priority; 2) The baseline situation in the region; 3) The actions implemented in the field of the investment priority; 4) The policy instruments used; 5) The indicators: Tables and Diagrams and 6) The impact analysis with conclusions. The consolidated report gets all the information entered in previous steps in a structured document that can be downloaded in PDF or MS Word format.

RIS3 Monitoring Rationale

Figure 2. RIS3 Monitoring Rationale

The steps of the application are the following:

•  The user imports data, as defined in the application 6.2 “Definition of output and results indicators” regarding the actions, investment priorities, measures, thematic objectives and output indicators (happens once and then in case policy changes)

•  The user imports data regarding the performance of the region (happens periodically and is optional)

•  Calculations and other data processes are conducted.

•  Tables and charts are generated.

•  The user can navigate through the results and modify graphical preferences or elements to be displayed.

•  The user starts elaborating the sections of the Monitoring report per investment priority.

•  The consolidated report regarding the region’s performance for a specific time is produced.

The data used to track and monitor the implementation process of the RIS3 strategy include information about:

•  The vision and priorities defined, including the objectives and planned actions set through the RIS3 process;

•  The performance of the region across time regarding the targets set in the previous step of the strategy (output and result indicators are used at this point).

The output of this application includes interactive dashboards providing information about the current performance of the region for certain time modes. Through the combination of innovative visualisation techniques with comparatives tables that depict changes of different measures across time, users can see the evolution of the overall strategic plan at a glance.

One the main benefits of the application is that users can use the application to formalise monitoring of the RIS3 implementation. All the data are stored in a relational database hence each user can retrieve the information easily at different time intervals.

The data used to track and monitor the implementation process of the RIS3 strategy include information about: The vision and priorities defined, including the objectives and planned actions set through the RIS3 process, The performance of the region across time regarding the targets set in the previous step of the strategy (output and result indicators are used at this point). The budget performance and the stakeholders (users) satisfaction of the RIS3 implementation from the surveys of the beneficiaries and ‘end-users’ as defined in their relative applications.

Each region can access solely their own data while data of the result indicators that are used in the correlation and multivariate linear regression analysis are gathered from Eurostat and are public available and open for any kind of use.

The following Figure 3 display the key benefits of the application that are considered to greatly support regional actors and stakeholders for the further development of their regions and the successful and cost effective implementation of RIS3.

Benefits to stakeholders when using the Application 6.1 RIS3 Monitoring

Figure 3. Benefits to stakeholders when using the Application 6.1 RIS3 Monitoring.

For the use of the Application 6.1 a valid user account is required. The application is web based and all data are stored in a MYSQL database.

The data that are required so that the users can work effectively are the following:

•  Output Indicators. The data source is application 6.2

•  Budget data. The data source is the Budgeting application of ONLINE S3 Platform

•  User satisfaction. The data source is Beneficiaries and end users’ satisfaction application

•  Balance scorecard: The data source is Balance scorecard application ONLINE S3 Platform

The required data are imported in the form of CSV, XLS, JSON files depending on the source application. The CSV columns are semi-column delimited (;) and cells can not include the semi-column in the data. The CSV rows are line feed delimited (\n).

The structure of the files are presented in the following tables:

•  Table 1 CSV file structure Output Indicators.

•  Table 2 CSV file structure Budget Indicators.

•  Table 3 XLS (XLSX) file structure User Satisfaction Indicators.

Each page in the “RIS3 Monitoring” 6.1 application has two main menu areas:

•  The first area includes two buttons: i) to create a new monitoring analytics and to view the list of each users’ monitoring analytics (Figure 5 My Monitoring analytics).

•  The second menu displays on the left side the name of the monitoring analytics that user is working on. On the right side, there are six (6) buttons with reports icons that the user can click to go to specific steps in the process of the application.

Main menu areas

Figure 4. Main menu areas

My monitoring analytics displays a list of all the strategies analytics that each user has created. Each user can create more than one monitoring analytics to refer to different time periods. The user can view / edit or delete each monitoring analytics by clicking on the respective links. The Complete / Incomplete status links show if the user for each report has submitted each step in the database. For a report to be complete all five steps have to submitted. The Process link drives the user in the following Figure 6 The process.

Figure 5. My Monitoring analytics

In this Process page the status of each step is highlighted. In the completed i.e. submitted steps the arrow in the green circle appears.

The process

Figure 6. The process

In the first step, the user must enter basic information to identify the monitoring analytics: Its name and referring geographical region. The last modification date field is automatically updated when then user saves any of the steps

Key report information

Figure 7. Key report information

The user must click on the button save and continue to store the entered information in the application / database.

This is the core step where the expert elaborates the sections of the Monitoring report. The report is structured per Investment Priority (IP). Its RIS3 strategy, doesn’t have to address all IPs hence experts should elaborate only the IPs that the RIS3 of the specific region addresses. The user must click on the button save and continue to store the entered information in the application / database.

Selection of Investment priority for RIS3 Monitoring report

Figure 8. Selection of Investment priority for RIS3 Monitoring report

After selecting the IP, from the drop-down list, the following subsections as shown in Figure 9 RIS3 Monitoring report per Investment priority, should be elaborated.

1. Objectives of the investment priority

2. Baseline situation in the region

3. Actions implemented in the field of the investment priority

4. Policy instruments used

5. Indicators: Tables and Diagrams

6. Impact analysis / Conclusions

In the provided editor, the expert (user) can upload images or insert tables of data, by clicking the appropriate buttons.

The user must click on the button save and continue to store the entered information in the application / database.

RIS3 Monitoring report per Investment priority

Figure 9. RIS3 Monitoring report per Investment priority

The expert can navigate to the following steps as “Output indicators dashboard”, “Budget dashboard” and “User Satisfaction dashboard” to get the data and visualizations for the subsection 5) Indicators: Tables and diagrams. Each of the steps are analyzed in following section of this guide.

In this section, the user can upload data in a CSV format coming from the 6.2 Output and result indicators application. The following screen shot, Figure 10 Output indicators dashboard, displays how the page will look like after data have been uploaded. The grey charts are a visual aid to display how the three charts will look alike.

To upload data into the dashboard you should first choose the CSV file and then click the “Upload” button. If the upload is successful and the CSV is parsed correctly the message “Upload has been successful. Number of rows: xx. Number of cols: xx” appears.

The user can create different combinations of charts by checking variables and years to be presented on the table.

Output indicators dashboard

Figure 10. Output indicators dashboard

The chart “Indicators per years” can be either in the form of vertical bars, or in the form of lines. This can be chosen by clicking the "Chart" link in the right side of the Indicators per years chart title.

Output indicators per years Vertical Bar Charts

Figure 11. Output indicators per years Vertical Bars Chart

The second chart is grouping the selected indicators and display their values in different years in horizontal bars. The user can either use the selected variables and years or choose other variables and years. Following the selection of variables the "Chart" link should be clicked. There is no limitation to the number of variables that can be presented in each chart.

Grouped per indicator chart

Figure 12. Grouped per indicator chart

Target completion chart

Figure 13. Target completion chart

The “Comments” text area refers to the Output indicators dashboard step.

The produced charts can be exported and downloaded in PNG format by clicking the link “Export”. To append the chart in the “Comments” text area the user can click on “Append” link. The chart is uploaded automatically to the server as a PNG file and following is appended in the end of the “Comments” text area.

To upload data into the dashboard you should first choose the XLS (XLSX) file and then click the “Upload” button. If the upload is successful and the XLS (XLSX) is parsed correctly the message “Upload has been successful.” appears.

All Results indicators

Figure 14. All Results indicators

The user must click on the button save and continue to store the data in the application / database.

To upload data into the dashboard you should first choose the XLS (XLSX) file and then click the “Upload” button. If the upload is successful and the XLS (XLSX) is parsed correctly the message "Upload has been successful." appears

User satisfaction dashboard

Figure 15. User satisfaction dashboard

User satisfaction dashboard legend

Figure 16. User satisfaction dashboard legends per charts

The user must click on the button save and continue to store the data in the application / database.

To upload data into the dashboard you should first choose the JSON file, that has been produced by the Application Balance Scorecard and then click the “Upload” button. If the upload is successful and the JSON is parsed correctly the message "Upload has been successful."

Balance satisfaction dashboard

Figure 17. Balance satisfaction dashboard

Balance satisfaction dashboard legend

Figure 18. Balance satisfaction dashboard legends per charts

The user must click on the button save and continue to store the data in the application / database.

Table 1 CSV file structure Output Indicators

Column Title

Description

type

Comments

a_index_code

Common Output (CO) indicator code

text

action_id

Action code

text

series_unit

CO indicator unit

text

action_title

The title of the action

text

action_to

The code of the thematic objective

text

This is set when actions are defined in the ONLINE S3 Platform Application .6.2

action_to_title

The title of the thematic objective

text

This is set when actions are defined in the ONLINE S3 Platform Application .6.2

action_challenges

The code of the investment priority

text

This is set when actions are defined in the ONLINE S3 Platform Application .6.2

action_ip_title

The title of the investment priority

text

This is set when actions are defined in the ONLINE S3 Platform Application .6.2

action_axis

The code of the measure related to the action

text

This is set when actions are defined in the ONLINE S3 Platform Application .6.2

action_axis_title

The title of the measure related to the action

text

title

The title of the CO indicator

text

target_value

The CO indicator target value

text

This is set when actions are defined in the ONLINE S3 Platform Application .6.2

target_year

The CO indicator target year

text

This is set when actions are defined in the ONLINE S3 Platform Application .6.2

erdf_cat

code of the ERDF category

text

Per CO indicator

erdf_title

Title of the ERDF category

text

Per CO indicator

reg_prof_cat

code of the Regional profile category

text

Per CO indicator

reg_prof_title

Title of the Regional profile category

text

Per CO indicator

Y2015

Value of the indicator for Ynnnn

Real Number

Year is defined in Application 6.2 when defining the action. Decimal point is dot

Y2016

Value of the indicator for Ynnnn

Real Number

Year is defined in Application 6.2 when defining the action. Decimal point is dot

Y2017

Value of the indicator for Ynnnn

Real Number

Year is defined in Application 6.2 when defining the action. Decimal point is dot

Y2018

Value of the indicator for Ynnnn

Real Number

Year is defined in Application 6.2 when defining the action. Decimal point is dot

Y2019

Value of the indicator for Ynnnn

Real Number

Year is defined in Application 6.2 when defining the action. Decimal point is dot

Y2020

Value of the indicator for Ynnnn

Real Number

Year is defined in Application 6.2 when defining the action. Decimal point is dot

Table 2 XLS file structure Budget Indicators

Column Title

Description

type

Comments

A

RIS3 priority

text

B

Measures

text

C

Programme

text

D

Objective

text

E

Intervention

text

F

Form of finance

text

G

Funding source

text

H

Comment

text

I

2017 - planned value

Number

J

2017 - committed value

Number

K

2017 - spent value

Number

L

2018 - planned value

Number

M

2018 - committed value

Number

N

2018 - spent value

Number

O

2019 - planned value

Number

P

2019 - committed value

Number

Q

2019 - spent value

Number

Table 3 XLS file structure User Satisfaction Indicators

Column Title

Description

type

Comments

A

Timestamp

date

B

1. Please choose one category to define your primary point of view in this survey, according to the “quadruple helix”

Option

C

2. Age category (years):

Option

D

3. Gender

Option

E

4. Years of work experience (years):

Option

F

5. Have you ever participated in any development phase regarding smart specialisation strategy?

Option

G

6. Have you ever participated in any implementation phase regarding smart specialisation strategy?

Option

H

7. Did you know about smart specialisation before this questionnaire?

Option

I

8. If yes, how did you get knowledge about smart specialisation?

Array of options

J

9. Were you familiar with the contents of our regional smart specialisation strategy before this questionnaire?

Option

K

10. If yes, how did you get to know about the contents of our smart specialisation strategy?

Array of options

L

11. Are you satisfied with the smart specialisation development process in our region?

Option

M

12. Why/why not?

text

N

13. Are you satisfied with the contents of our regional smart specialisation strategy as a whole?

Option

O

14. Why/why not?

text

P

How satisfied are you with the chosen smart specialisation domains? [15. Priority area 1]

Option

Q

How satisfied are you with the chosen smart specialisation domains? [16. Priority area 2]

Option

R

How satisfied are you with the chosen smart specialisation domains? [17. Priority area 3]

Option

S

18. Are you satisfied with the implementation of our regional smart specialisation strategy as a whole?

Option

T

19. Why/why not?

text

U

How satisfied are you with the implementation of the following smart specialisation priority areas? [20. Priority area 1]

Option

V

How satisfied are you with the implementation of the following smart specialisation priority areas? [21. Priority area 2]

Option

W

How satisfied are you with the implementation of the following smart specialisation priority areas? [22. Priority area 3]

Option

X

23. Do you work in an area that is related to one of the chosen smart specialisation domains?

Option

Y

24. If yes, which one(s)?

Array of options

Z

25. What has been the impact of smart specialisation implementation in the the above domain so far?

Option

AA

26. What has been the key effects of smart specialisation on our region? (you may select several)

Array of options

AB

27. Have your organization been affected by smart specialisation strategy so far?

Option

AC

28. If yes, how?

text

AD

29. Do you have any suggestions how to improve the smart specialisation strategy process?

Option

AE

30. If yes, please specify:

text

AF

31. Do you have any suggestions for new priority areas in our region?

Option

AG

If yes, which area and why it is important?

text

AH

Do you have any suggestion of new activities to boost innovation in the region?

Option

AI

If yes, please specify

test