Mendeley Indicator Reports
Here are the steps necessary to collect Mendeley reader data and calculate a range of indicators for a collection of publications, including the Mean Normalised Log-transformed Citation Score (MNLCS) and the Normalised Proportion Cited (EMNPC).
- Step 1: Identify the group of publications to be assessed and categorise them by field (e.g., using Scopus or WoS subject categories).
- Step 2: Save the article information (authors, title, journal, publication year) in a standard tab-delimited format in a separate file for each subject category/year combination. First, discard publications that are in small subject/year combinations (e.g., <100 publications). Create tab-delimited files for the each subject/year. There should be one line per publication. Each line should contain the author names in standard format (following Scopus or Web of Science formats would be ideal), the publication year, the article title and the journal name (ignore this for books). The first line of the file should contain header information. Here is an example of the format for journal articles and for books. If your data is in a spreadsheet, it can be saved in this format using the Save As command and selecting the Plain text (tab delimited) format. The filename for each file must contain the subject name and year, and end with -[group].txt, where [group] should be replaced by a name for the collection of articles. The same [group] should be used for files containing publications from the same group. If the files are in Scopus of the Web of Science then choose the tab delimited format in which to save them.
- Step 3: For each retained subject/year combination, a benchmarking sample is needed of articles from the rest of the world. For this, download all articles from the Scopus/WoS (if possible) field/year or a large balanced sample (e.g., the first and last 5000 articles published in the category) for the world reference set. Filter out any large trade or art journals with a high proportion of uncited articles. Name the files using the standard Webometric Analyst naming convention so that each filename contains the subject name and year, and ends with -world.txt. These filenames must exactly match the group filenames, except for replacing -[group].txt with -world.txt. All of the files should be stored within a single folder that does not contain any other files. The figure below shows four years, three fields and three different groups (MRC, Wellcome and NIH) all in the correct filename format.
- Here is a small artificial example of a complete set of files, with all publications in a single file being from the same field and year.
- Step 4: Use Webometric Analyst to gather the Mendeley data. For, this, start Webometric Analyst, click the Mendeley tab, tick the box Repeat for all files in the same folder, click the Search for Publications (v1) button and select one of the group or world tab delimited files. Follow the instructions to enter a Mendeley key. This will create many extra files in the folder, an extract of which is shown below.
- Example of Mendeley output files.
- Step 5: Use Webometric Analyst to calculate MNLCS and EMNPC and confidence limits for both. For this, start Webometric Analyst and select Calculate MNLCS and EMNPC for a set of *Mendeley API* results (structured file names) from the Reports menu. Select the folder containing all of the files, when requested. This will create two new files. The file called all_data.txt, contains all of the data extracted from the searches in a format that can be loaded into a stats package or spreadsheet. This is a backup file in case you want to calculate your own indicators. The file called report.txt contains MNLCS and EMNPC values for each individual file in a long list at the top. Near the end of the file it then reports tables of the combined MNLCS and EMNPC values for the whole collection. [see below for a sample report]
- Step 6: If you want MNLCS and EMNPC calculated separately for each year, then create new folders, one for each year, and copy all the files from each year into the relevant year folder. Repeat the step above for each year folder.
Source of indicator data: C:\Users\Public\Documents\data\Mendeley readers structured names Total number of world files (e.g., one per field and year): 2
The next section of this report gives information for individual files. Scroll to the bottom of the report for the main results. Note that EMNPC=MNPC for individual files.
World File: Biochemistry Molecular Biology Alcohol 2012 world_pubsFound_total85 Records : 500 Arithmetic mean of raw data : 16.844000 Geometric mean (95%CI) of raw data : 7.991652 (7.036911, 9.059811) Mean (95%CI) of ln(1+raw data) : 2.196297 (2.084045, 2.308548) Proportion non-zero (95%CI) : 0.836000 (0.801005, 0.865871) MNLCS - mean (95%CI) of world normalised ln(1+raw data) [population version] : 1.000000 (0.948890, 1.051110) MNLCS - mean (95%CI) of world normalised ln(1+raw data) [sample version] : 1.000000 (0.930197, 1.075041) EMNPC - world normalised proportion cited (non-zero) (95%CI): 1.000000 (0.946767, 1.056227)
Group file: Spain. In set: Biochemistry Molecular Biology Alcohol 2012 Records : 193 Arithmetic mean : 16.663212 Geometric mean (95%CI) of raw data : 8.162284 (6.598901, 10.047313) Mean (95%CI) of ln(1+raw data) : 2.215095 (2.028004, 2.402187) Proportion non-zero (95%CI) : 0.808290 (0.746954, 0.857593) MNLCS - mean (95%CI) of world normalised ln(1+raw data) [population version] : 1.008559 (0.923374, 1.093744) MNLCS - mean (95%CI) of world normalised ln(1+raw data) [sample version] : 1.008559 (0.911468, 1.110933) EMNPC - world normalised proportion cited (non-zero) (95%CI): 0.966854 (0.893995, 1.045651)
World File: Chemistry Alcohol 2012 world_pubsFound_total85 Records : 500 Arithmetic mean of raw data : 12.230000 Geometric mean (95%CI) of raw data : 4.467472 (3.847160, 5.167167) Mean (95%CI) of ln(1+raw data) : 1.698816 (1.578393, 1.819240) Proportion non-zero (95%CI) : 0.698000 (0.656372, 0.736608) MNLCS - mean (95%CI) of world normalised ln(1+raw data) [population version] : 1.000000 (0.929113, 1.070887) MNLCS - mean (95%CI) of world normalised ln(1+raw data) [sample version] : 1.000000 (0.904422, 1.105679) EMNPC - world normalised proportion cited (non-zero) (95%CI): 1.000000 (0.921877, 1.084743)
Group file: Spain. In set: Chemistry Alcohol 2012 Records : 282 Arithmetic mean : 9.719858 Geometric mean (95%CI) of raw data : 4.239215 (3.490230, 5.113132) Mean (95%CI) of ln(1+raw data) : 1.656172 (1.501904, 1.810439) Proportion non-zero (95%CI) : 0.680851 (0.624327, 0.732514) MNLCS - mean (95%CI) of world normalised ln(1+raw data) [population version] : 0.974897 (0.884089, 1.065706) MNLCS - mean (95%CI) of world normalised ln(1+raw data) [sample version] : 0.974897 (0.865313, 1.094329) EMNPC - world normalised proportion cited (non-zero) (95%CI): 0.975431 (0.884203, 1.076072)
The table below contains the same information as above and can be cut and pasted into a spreadsheet for convenience. ====================================================================================================== Set (e.g.,Field/Year) Group Records Arithmetic mean of raw data Proportion non-zero (95%CI) Lower95 Upper95 Mean of ln(1+raw data) (95%CI) Lower95 Upper95 Geometric mean (95%CI) of raw data Lower95 Upper95 MNLCS - mean (95%CI) of world normalised ln(1+raw data) Lower95Sample Upper95Sample Lower95Population Upper95Population EMNPC - world normalised proportion cited (non-zero) Lower95 Upper95 Biochemistry Molecular Biology Alcohol 2012 world_pubsFound_total85 World 500 16.844000 0.836000 0.801005 0.865871 2.196297 2.084045 2.308548 7.991652 7.036911 9.059811 1.000000 0.930197 1.075041 0.948890 1.051110 1.000000 0.946767 1.056227 Biochemistry Molecular Biology Alcohol 2012 world_pubsFound_total85 Spain 193 16.663212 0.808290 0.746954 0.857593 2.215095 2.028004 2.402187 8.162284 6.598901 10.047313 1.008559 0.911468 1.110933 0.923374 1.093744 0.966854 0.893995 1.045651 Chemistry Alcohol 2012 world_pubsFound_total85 World 500 12.230000 0.698000 0.656372 0.736608 1.698816 1.578393 1.819240 4.467472 3.847160 5.167167 1.000000 0.904422 1.105679 0.929113 1.070887 1.000000 0.921877 1.084743 Chemistry Alcohol 2012 world_pubsFound_total85 Spain 282 9.719858 0.680851 0.624327 0.732514 1.656172 1.501904 1.810439 4.239215 3.490230 5.113132 0.974897 0.865313 1.094329 0.884089 1.065706 0.975431 0.884203 1.076072 ======================================================================================================
The Mean Normalised Log-transformed Citation Scores (MNLCS) in the table below are the best to use to compare the group overall with the world average if there are multiple different world averages (e.g., different fields and/or years). For each group they are the average of ln(1+c) values, divided by the world average ln(1+c) for the file (e.g., field and year). The world average MNLCS should always be 1. MNLCS values above 1 indicate that the group average is higher than the world average; MNLCS values below 1 indicate that the group average is lower than the world average. WARNING! MNLCS POPULATION confidence limits below are optimistic because they do not take into account the variability in the world average value. - Please use only the MNLCS SAMPLE confidence limits. These are adjusted from the population limits using the weighted average Feiller Expansion calculation. - NaN in the sample confidence limits mean that these are impossible to calculate and are effectively infinite. ====================================================================================================== Group SampleSize MNLCS Lower95Sample Upper95Sample Lower95Population Upper95Population World 1000 1 0.940734 1.064616 0.95632661924876 1.04367338075124 Spain 475 0.988574809656072 0.913058 1.069826 0.924552544285857 1.05259707502629 ======================================================================================================
Proportion non-zero calculations - these are *biased* estimators and should normally be ignored because different fields and years can have different natural proportions of cited articles. ====================================================================================================== Group RawData_N RawData_Proportion_Nonzero RawData_Lower95 RawData_Upper95 world 1000 0.767000 0.739807 0.792149 Spain 475 0.732632 0.691080 0.770451 ======================================================================================================
Field equalised proportion non-zero calculations and EMNPC - all group sample sizes are set to the arithmetic mean sample size for sets with at least one publication. ====================================================================================================== Group N AvProportionNonzero Lower95 Upper95 EMNPC Lower95 Upper95 world 1000 0.767000 0.739807 0.792149 1.000000 0.952902 1.049426 Spain 475 0.744571 0.703499 0.781719 0.970757 0.911822 1.033502 ======================================================================================================
MNPC calculations (don't use) - similar to the above. Confidence intervals are the weighted average of the confidence intervals for each individual field/year set. ====================================================================================================== Group N MNPC Lower95 Upper95 MNPCLower95boot MNPCUpper95boot EMNPCLower95boot EMNPCUpper95boot world 1000 1.000000 0.934322 1.070485 1.000000 1.000000 1.000000 1.000000 Spain 475 0.971946 0.888182 1.063712 0.907687 1.037393 0.910455 1.031518 ======================================================================================================