Bibliometric Analysis Explained: What It Can and Cannot Tell You

Bibliometric analysis applies quantitative methods to publication data to describe the structure and development of a research field. Typical outputs are counts of publications over time, the most productive authors, institutions and countries, the most cited papers, networks of co-authorship, co-citation and keyword co-occurrence, and clusters of related topics. In engineering research it is used to map emerging areas, to support literature reviews and to identify gaps. It describes the literature; it does not by itself establish the quality or truth of the work in it.

Bibliometric databases differ in coverage, fields, indexing policy and data quality. Some cover more journals and conferences in engineering, some have better citation linking, and some offer open data. Records exported from different databases use different formats and identifiers, and merging them requires deduplication. The choice should be justified, and the limitations stated, such as language bias and incomplete coverage of conference papers or books.

Document the database, the date of the search, the query and any filters.

The search query determines everything that follows. Build it from the research question, using synonyms, spelling variants and truncation, and combine terms with logical operators. Test the query against papers known to be relevant and check for irrelevant results caused by ambiguous terms. Decide on the time window, document types and languages. Record the number of records at each step.

A narrow query misses relevant work; a broad one adds noise. Iteration with expert judgement is normal.

Clean author names, affiliations, keywords and sources. Merge variants of the same author or institution cautiously, because errors distort networks. Standardise keywords, for example by merging singular and plural forms or synonyms, using a documented thesaurus. Remove duplicates and records outside scope. Save the raw and cleaned files.

Cleaning is time-consuming and often the largest source of error, so it should be reported.

Performance analysis reports productivity and impact: publications, citations and indicators such as the h-index for groups or sources. Science mapping examines relationships: co-citation analysis links papers cited together, bibliographic coupling links papers that share references, co-authorship maps collaboration, and keyword co-occurrence shows topics. Clusters are identified using algorithms, and labelled by reading the contents.

Parameter choices, such as thresholds for minimum occurrences or clustering resolution, affect the maps and must be reported.

Tools exist for network visualisation and for analysis in common programming languages. Whatever the software, record the version and the settings, and share the data and scripts if licences permit. A reader should be able to reproduce the figures from the dataset.

Mention of a tool is not an endorsement.

Citation counts depend on field, age, document type and database. Highly cited papers are not necessarily better. Clusters are products of algorithms and may merge or split topics. Trends in publication counts reflect changes in database coverage, in the size of the research community and in terminology. Avoid causal claims such as that a topic is declining merely because publication numbers drop in recent years, since indexing delays affect the latest year.

Combine bibliometric findings with a reading of key papers.

Bibliometric analysis and systematic review answer different questions. Bibliometrics maps the field quantitatively; a systematic review synthesises evidence from a defined set of studies. They can be combined: a bibliometric map can inform the scope and search of a review, and a review can interpret clusters. Combining them requires clarity about which part serves which purpose.

Our manuscript services include support for systematic reviews with bibliometric analysis.

Describe the question, database, query, dates, inclusion criteria, cleaning, indicators, software and parameters. Present figures with legible labels and state how maps were generated. Discuss limitations, such as database coverage and keyword dependence. Provide the dataset where possible.

Journals vary in their expectations, so check recent bibliometric papers in the target journal.

We support the design of the search, the cleaning and analysis workflow, the interpretation of maps and the writing of the manuscript. We do not fabricate data or cite unread papers, and we ask authors to verify all references.

Suppose a team wants to map research on additive manufacturing of metals. Step one: state the questions, such as how the literature grew, which topics dominate and which countries collaborate. Step two: choose databases and build a query combining process terms with material terms. Step three: export records with citations and references, then deduplicate. Step four: clean keywords and affiliations. Step five: compute growth and productivity indicators. Step six: build keyword and co-citation maps and name clusters after reading representative papers. Step seven: interpret and write, with limitations.

This outline is illustrative and does not describe any client project. It shows the logic: question, data, cleaning, analysis, interpretation, limitations.

Frequent errors include overly broad or narrow queries, unreported cleaning, treating citation counts as quality, over-interpreting cluster labels, using different time windows for different analyses, ignoring database coverage, and presenting maps without parameter details. Another error is claiming emerging trends from the last year's counts, which are incomplete.

A peer reviewer will ask for the query, the dates and the parameters; supply them up front.

Bibliometric indicators can be misused to evaluate individuals. Keep analysis at the level of fields, topics and aggregates, and avoid naming individuals in a way that implies evaluation unless it is the aim and is handled responsibly. Respect licensing of data, and cite the databases and tools properly.

Avoid techniques that inflate citations, such as coercive citation.

Content analysis of selected papers, topic modelling of abstracts, patent analysis and altmetrics give additional perspectives. Each has limitations: topic models depend on preprocessing and the number of topics; patent data have different dynamics; altmetrics measure attention rather than quality. Using several methods and triangulating results strengthens conclusions.

Describe each method and its parameters.

Which database should I use? One whose coverage suits your field, with the choice justified; using two and merging is common. How many records are enough? Enough to represent the field; a few hundred can support a focused map, thousands a broad one, but quality matters more than quantity. Can I rely on automatic clustering? Treat clusters as hypotheses and check them by reading. Do I need a protocol? A written plan helps, particularly if combined with a systematic review.

Is bibliometric analysis a review? It is a form of science mapping; some journals accept it as a research article if the method and interpretation are rigorous.

Bibliometric analysis offers a structured way to see a field: its growth, its communities and its themes. Its value depends on a transparent query, careful cleaning, suitable indicators and cautious interpretation. Report enough detail for others to reproduce the work, and combine the maps with expert reading.

Match indicators to questions. For growth, use publication counts per year with a note on indexing delay. For influence, use field-normalised citation measures if available, or at least compare within the same year and document type. For collaboration, use co-authorship networks with thresholds. For topics, use keyword or term co-occurrence with a stated minimum frequency. Avoid reporting indicators because the software provides them.

Each indicator should have a sentence stating what it is and what it is not.

Network maps are attractive but easily misread. Label nodes legibly, explain what node size, edge thickness and colour mean, state layout and clustering settings and avoid overcrowding by filtering low-frequency items. Add tables for the most important items, and consider a time overlay to show evolution. Provide high-resolution figures and a legend in plain language.

Readers should be able to understand the main message of a map without reading the software documentation.

Write your questions, test two or three search queries in a database, compare the results with known key papers and decide on thresholds. Then export, clean and analyse, recording each step. Ask a colleague to review the query.

A bibliometric study is only as good as its query, data cleaning and interpretation. Document the database and dates, report parameters, treat clusters as hypotheses and combine maps with expert reading.

Used carefully, it reveals the shape of a field; used carelessly, it produces attractive but unreliable figures.