Center for Refining Information (CRI) is a spin-off company founded by the University of Warsaw in 2019

CRI commercializes the innovative achievements of the Faculty of Journalism, Information and Bibliology in the field of analysis of large information resources (BigData).

Studies carried out since 2010 in the Department of Information Technology of the Media have resulted in the development of a concept and practical implementation of a new innovative system for searching, collecting and analyzing large text collections called Refining Information (RI). The effectiveness of this method was confirmed, among others, by the results of research on electoral preferences, indicating social expectations or identifying and determining the dynamics of changes in innovation processes. The scope of CRI services includes an assessment of the current state of the studied cases and forecasts of social, political and economic processes. The researchers are also in the CRI market offer.

The strength of CRI is the staff of over a dozen people: IT specialists, analysts, economists and bibliologists. The unique, transparent ICT tools created by CRI experts enable reliable, efficient identification and collection of source data as well as their precise quantitative analysis – refining information enriched by AI elements, especially the machine lerning.

We are technologically independent, the system is scalable, redundant – with sufficient power. Technical facilities meet the basic conditions of teleinformation security (staff – state certificate of the inspector and IT security administrator).

One of the commercial projects which has been implemented by CRI was the development of a unique system for monitoring technological trends (MTT).The system has functionalities that allow (1) automatic collection and standardization of source materials from indicated online and offline sources. Another MTT functionality is (2) identification – with the use of AI (machine learning) tools – events / phenomena describing technological trends. After beeing identified, the trends are subjected to (3) a quantitative assessment of their change and prediction dynamics. This task is carried out on the basis of analyzes of large volume data, covering the problems of global and Polish trends in the field of technology. These evaluations have been verified in several projects made for the National Center for Research and Development. The current state and dynamics of technological innovation changes were examined in such the fields like: energy, aviation, cloud computing or photonic material technologies.

The results of these studies are effectively used to support R & D and innovation programs, based on reliable and, above all, up-to-date, self-accumulated, data (on a global scale) that allow to identify and forecast technological trends. The obtained data can be used to (4) identify long-term trends, exceptional events in the scope of: market expectations, competition activities, directions of research and development and innovation development to objectively describe what is currently happening on the market, how trends shaped Poland and the world, and whether the areas of innovation identified today will be valid in the near future. The tools created for refining information have proved useful also in research in the field of social, political and technical sciences which was confirmed by the documented applications (5) of refining information, using sentiment analyzes, including in terms of quantitative assessment of the course and accurate prediction of parliamentary and presidential elections in Poland in 2010/2011.

(http://studiamedioznawcze.pl/Numery/2013_2_53/gogolek-en.pdf; http://studiamedioznawcze.pl/Numery/2013_3_54/gogolek-en.pdf) and 2014/2015 (http://studiamedioznawcze.pl/Numery/2016_4_67/gogolek-en.pdf).

1. Identification and collection of source materials.

Acquiring the information in question. Search and resource identification strategies. The distribution of sources according to the categories. Methodology for identifying thematic sources of information.

2. Exploration of large, variable and diverse data sets identified. Data collection procedure, data formats, categorization. Classification of source documents.

3. Analytical tools. Classification of source materials.

4. Classification method. Classification by source category. Data classification. Training set. Machine learning. Processing of input data. Attribute analysis. Division into teaching and test set. Model learning. Model verification. Category mapping.

5. Analytical tools. Identification of technological trends. The functioning algorithm. Identification of technological trends. Classification Poland – world. Information Osmosis. Identification of attributes. Data cleansing. Tokenization. Quantitative analysis. Statistics TF-IDF. Bigram analysis. Cluster points.

The examples below presents how Refining Information technology can be aplicable to analyse current international events:

“Islam” vs. “ISIS”

Occurrences of the terms „ISIS” and „Islam” were analyzed in the period of study corresponding to the area of interest of Al Jazeera.

Figure 1: The area of particular interest of the Al Jazeera TV network

It is clear that the two names are linked with the focus on the Islamic State. The relationship between them is interesting. The stereotypical interpretation that formed the dominant narrative in European countries, especially in relation to refugees, identified “Islam” and its followers with “ISIS.” In Poland, for example, some media channels claimed that Islam is a threat to the security of citizens and linked it to the terrorism perpetrated by the Islamic State. However, statistics do not support this claim. In the period from 2017-04-17 to 2017-07-03, there was a clear rise in the occurrence of the term “ISIS” on websites related to Al Jazeera. It is worth noting that until May 29, 2017, both the terms “ISIS” and “Islam” were almost on an equal footing in terms of use—their frequencies overlapped in the news channels narrowed down to a weekly period (Table 1).

Table 1: Occurrence of the terms “ISIS” and “Islam” from 2017-04-17 to 2017-07-03 on Al Jazeera websites

(Source: own study)

A significant change in the trend occurred in the beginning of June 2017. The frequency of the term “Islam” decreased and that of “ISIS” increased significantly (Figure 2). In the last 2 months of the period studied, “Islam” did not exceed 26 occurrences while “ISIS” occurred nearly six times more often. Until May 29, “Islam” appeared on average 25.6 times a week and “ISIS” 25.4 times, but from June 5 to July 3, “Islam” appeared on average 13.5 times while “ISIS” occurred 75.5 times.

[gdoc key=”https://docs.google.com/spreadsheets/d/1PuI66DichANXorQH5Ub3hAUS8_pZHUrdANtQ8djBHZ8/edit?usp=sharing” chart=”Line” title=”Figure 2: Changes in trends of occurrence of terms “ISIS” and “Islam” from 2017-04-17 to 2017-07-03 (Source: our study)”]

Figure 2: Changes in trends of occurrence of terms “ISIS” and “Islam” from 2017-04-17 to 2017-07-03 (Source: our study)

Finally, the relation between these terms was determined using the correlation between their numbers of occurrence during the studied period (Table 2). The correlation was negative, which means that a more frequent occurrence of the term “ISIS” was accompanied by a decrease in the frequency of use of the term “Islam.” The relationship was not strong, and a value of −0.226407 suggests that these terms were statistically independent. Thus, hypothesis H1 was considered to have been verified.

Table 2: Correlation between the occurrence of the terms “ISIS” and “Islam” from 2017-04-17 to 2017-07-03

(Source: own study)

“Saud” and “ISIS” vs. Oil Prices

Using correlation as a measure of dependence, of the terms used in the analyzed information channel, those that were statistically significantly related to the price of crude oil were highlighted. The terms “Saud” and “ISIS” were chosen as significant (Table 3). The collected data were aggregated into weekly intervals to create cross-channels in the information stream.

Table 3: The terms “Saudi” and “ISIS,” and the price of crude oil in the studied period

(Source: https://www.bankier.pl/inwestowanie/profile/quote.html?symbol=ROPA accessed: 2017-09-22), own study)

Both terms had a strong positive relationship with oil prices: the correlation coefficient for “ISIS” and “Saud” was 0.86 (Table 4). On the contrary, oil prices were correlated negatively with the occurrence of “ISIS,” −0.79, and “Saud,” −0.64.

Table 4: Correlation between the occurrence of “ISIS,” and “Saud” and oil prices

(Source: own study)

The results led to the conclusion that the frequency of occurrence of these terms can be used to forecast oil prices. Multiple regression analysis was used to make price predictions, where occurrences of the terms „Saudi” and “ISIS” were treated as independent variables, and the price of oil was treated as a dependent variable. The multiple regression equation took the form of a linear equation:

1. OIL_c = a₁*S_c + a₂*I_c +a₀

where

OIL_c – price of crude oil in cross-channel c

S_c – number of occurrences of the term “Saud” in cross-channel c

I_c – number of occurrences of the term “ISIS” in cross-channel c

a_1, a_2, a₀ – variable coefficients

The coefficients of the equation below were then obtained (Table 5).

Table 5: Coefficients of regression equation

(Our own calculations)

That is,

a₁ = 0.00536

a₂ = −0.07441

a₀ = 54.05774

Following the calculations using the obtained coefficients and known values of the independent variables (“Saud,” “ISIS”) and the dependent variable („OIL”), we obtained the values presented in Figure 3.

USD

Figure 3: Oil price quotes and forecasted oil prices (multiple regression) (our own calculations)

The results can be considered satisfactory. The theoretically calculated oil price was within the range of ± 5% of the actual price. Therefore, by knowing the number of occurrences of the terms “Saud” and “ISIS,” we can forecast oil prices. Thus, hypothesis H2 can be considered to have been positively verified.

Terrorism vs. Acts of Terror

In the last stage of the analysis, the number of terms related to terrorism were collected and statistics were generated for each day. The data were compared with the number of victims (killed and injured) of terrorist acts that took place during the studied period (Table 6).

The terrorist events covered in the study are listed in Table 7.

Table 6: Occurrences of terms related to terrorism and the number of victims of terrorist acts in the studied period

Table 7: Terrorist attacks and the number of victims of (killed and injured) during the studied period

In Figure 4, we observe a worrying trend whereby terrorist attacks were accompanied by a decreasing occurrence of terms related to terrorism in the analyzed information stream (using data collected from sources related to Al Jazeera). Thus, hypothesis H3 was considered to have been verified.

Figure 4: Occurrences of terms related to terrorism and the number of victims of terrorist acts