problem solving scientific paper

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

Identifying problems and solutions in scientific text

Kevin heffernan.

Department of Computer Science and Technology, University of Cambridge, 15 JJ Thomson Avenue, Cambridge, CB3 0FD UK

Simone Teufel

Research is often described as a problem-solving activity, and as a result, descriptions of problems and solutions are an essential part of the scientific discourse used to describe research activity. We present an automatic classifier that, given a phrase that may or may not be a description of a scientific problem or a solution, makes a binary decision about problemhood and solutionhood of that phrase. We recast the problem as a supervised machine learning problem, define a set of 15 features correlated with the target categories and use several machine learning algorithms on this task. We also create our own corpus of 2000 positive and negative examples of problems and solutions. We find that we can distinguish problems from non-problems with an accuracy of 82.3%, and solutions from non-solutions with an accuracy of 79.7%. Our three most helpful features for the task are syntactic information (POS tags), document and word embeddings.

Introduction

Problem solving is generally regarded as the most important cognitive activity in everyday and professional contexts (Jonassen 2000 ). Many studies on formalising the cognitive process behind problem-solving exist, for instance (Chandrasekaran 1983 ). Jordan ( 1980 ) argues that we all share knowledge of the thought/action problem-solution process involved in real life, and so our writings will often reflect this order. There is general agreement amongst theorists that state that the nature of the research process can be viewed as a problem-solving activity (Strübing 2007 ; Van Dijk 1980 ; Hutchins 1977 ; Grimes 1975 ).

One of the best-documented problem-solving patterns was established by Winter ( 1968 ). Winter analysed thousands of examples of technical texts, and noted that these texts can largely be described in terms of a four-part pattern consisting of Situation, Problem, Solution and Evaluation. This is very similar to the pattern described by Van Dijk ( 1980 ), which consists of Introduction-Theory, Problem-Experiment-Comment and Conclusion. The difference is that in Winter’s view, a solution only becomes a solution after it has been evaluated positively. Hoey changes Winter’s pattern by introducing the concept of Response in place of Solution (Hoey 2001 ). This seems to describe the situation in science better, where evaluation is mandatory for research solutions to be accepted by the community. In Hoey’s pattern, the Situation (which is generally treated as optional) provides background information; the Problem describes an issue which requires attention; the Response provides a way to deal with the issue, and the Evaluation assesses how effective the response is.

An example of this pattern in the context of the Goldilocks story can be seen in Fig.  1 . In this text, there is a preamble providing the setting of the story (i.e. Goldilocks is lost in the woods), which is called the Situation in Hoey’s system. A Problem in encountered when Goldilocks becomes hungry. Her first Response is to try the porridge in big bear’s bowl, but she gives this a negative Evaluation (“too hot!”) and so the pattern returns to the Problem. This continues in a cyclic fashion until the Problem is finally resolved by Goldilocks giving a particular Response a positive Evaluation of baby bear’s porridge (“it’s just right”).

Example of problem-solving pattern when applied to the Goldilocks story.

Reproduced with permission from Hoey ( 2001 )

It would be attractive to detect problem and solution statements automatically in text. This holds true both from a theoretical and a practical viewpoint. Theoretically, we know that sentiment detection is related to problem-solving activity, because of the perception that “bad” situations are transformed into “better” ones via problem-solving. The exact mechanism of how this can be detected would advance the state of the art in text understanding. In terms of linguistic realisation, problem and solution statements come in many variants and reformulations, often in the form of positive or negated statements about the conditions, results and causes of problem–solution pairs. Detecting and interpreting those would give us a reasonably objective manner to test a system’s understanding capacity. Practically, being able to detect any mention of a problem is a first step towards detecting a paper’s specific research goal. Being able to do this has been a goal for scientific information retrieval for some time, and if successful, it would improve the effectiveness of scientific search immensely. Detecting problem and solution statements of papers would also enable us to compare similar papers and eventually even lead to automatic generation of review articles in a field.

There has been some computational effort on the task of identifying problem-solving patterns in text. However, most of the prior work has not gone beyond the usage of keyword analysis and some simple contextual examination of the pattern. Flowerdew ( 2008 ) presents a corpus-based analysis of lexio-grammatical patterns for problem and solution clauses using articles from professional and student reports. Problem and solution keywords were used to search their corpora, and each occurrence was analysed to determine grammatical usage of the keyword. More interestingly, the causal category associated with each keyword in their context was also analysed. For example, Reason–Result or Means-Purpose were common causal categories found to be associated with problem keywords.

The goal of the work by Scott ( 2001 ) was to determine words which are semantically similar to problem and solution, and to determine how these words are used to signal problem-solution patterns. However, their corpus-based analysis used articles from the Guardian newspaper. Since the domain of newspaper text is very different from that of scientific text, we decided not to consider those keywords associated with problem-solving patterns for use in our work.

Instead of a keyword-based approach, Charles ( 2011 ) used discourse markers to examine how the problem-solution pattern was signalled in text. In particular, they examined how adverbials associated with a result such as “thus, therefore, then, hence” are used to signal a problem-solving pattern.

Problem solving also has been studied in the framework of discourse theories such as Rhetorical Structure Theory (Mann and Thompson 1988 ) and Argumentative Zoning (Teufel et al. 2000 ). Problem- and solutionhood constitute two of the original 23 relations in RST (Mann and Thompson 1988 ). While we concentrate solely on this aspect, RST is a general theory of discourse structure which covers many intentional and informational relations. The relationship to Argumentative Zoning is more complicated. The status of certain statements as problem or solutions is one important dimension in the definitions of AZ categories. AZ additionally models dimensions other than problem-solution hood (such as who a scientific idea belongs to, or which intention the authors might have had in stating a particular negative or positive statement). When forming categories, AZ combines aspects of these dimensions, and “flattens” them out into only 7 categories. In AZ it is crucial who it is that experiences the problems or contributes a solution. For instance, the definition of category “CONTRAST” includes statements that some research runs into problems, but only if that research is previous work (i.e., not if it is the work contributed in the paper itself). Similarly, “BASIS” includes statements of successful problem-solving activities, but only if they are achieved by previous work that the current paper bases itself on. Our definition is simpler in that we are interested only in problem solution structure, not in the other dimensions covered in AZ. Our definition is also more far-reaching than AZ, in that we are interested in all problems mentioned in the text, no matter whose problems they are. Problem-solution recognition can therefore be seen as one aspect of AZ which can be independently modelled as a “service task”. This means that good problem solution structure recognition should theoretically improve AZ recognition.

In this work, we approach the task of identifying problem-solving patterns in scientific text. We choose to use the model of problem-solving described by Hoey ( 2001 ). This pattern comprises four parts: Situation, Problem, Response and Evaluation. The Situation element is considered optional to the pattern, and so our focus centres on the core pattern elements.

Goal statement and task

Many surface features in the text offer themselves up as potential signals for detecting problem-solving patterns in text. However, since Situation is an optional element, we decided to focus on either Problem or Response and Evaluation as signals of the pattern. Moreover, we decide to look for each type in isolation. Our reasons for this are as follows: It is quite rare for an author to introduce a problem without resolving it using some sort of response, and so this is a good starting point in identifying the pattern. There are exceptions to this, as authors will sometimes introduce a problem and then leave it to future work, but overall there should be enough signal in the Problem element to make our method of looking for it in isolation worthwhile. The second signal we look for is the use of Response and Evaluation within the same sentence. Similar to Problem elements, we hypothesise that this formulation is well enough signalled externally to help us in detecting the pattern. For example, consider the following Response and Evaluation: “One solution is to use smoothing”. In this statement, the author is explicitly stating that smoothing is a solution to a problem which must have been mentioned in a prior statement. In scientific text, we often observe that solutions implicitly contain both Response and Evaluation (positive) elements. Therefore, due to these reasons there should be sufficient external signals for the two pattern elements we concentrate on here.

When attempting to find Problem elements in text, we run into the issue that the word “problem” actually has at least two word senses that need to be distinguished. There is a word sense of “problem” that means something which must be undertaken (i.e. task), while another sense is the core sense of the word, something that is problematic and negative. Only the latter sense is aligned with our sense of problemhood. This is because the simple description of a task does not predispose problemhood, just a wish to perform some act. Consider the following examples, where the non-desired word sense is being used:

• “Das and Petrov (2011) also consider the problem of unsupervised bilingual POS induction”. (Chen et al. 2011 ).
• “In this paper, we describe advances on the problem of NER in Arabic Wikipedia”. (Mohit et al. 2012 ).

Here, the author explicitly states that the phrases in orange are problems, they align with our definition of research tasks and not with what we call here ‘problematic problems’. We will now give some examples from our corpus for the desired, core word sense:

• “The major limitation of supervised approaches is that they require annotations for example sentences.” (Poon and Domingos 2009 ).
• “To solve the problem of high dimensionality we use clustering to group the words present in the corpus into much smaller number of clusters”. (Saha et al. 2008 ).

When creating our corpus of positive and negative examples, we took care to select only problem strings that satisfy our definition of problemhood; “ Corpus creation ” section will explain how we did that.

Corpus creation

Our new corpus is a subset of the latest version of the ACL anthology released in March, 2016 1 which contains 22,878 articles in the form of PDFs and OCRed text. 2

The 2016 version was also parsed using ParsCit (Councill et al. 2008 ). ParsCit recognises not only document structure, but also bibliography lists as well as references within running text. A random subset of 2500 papers was collected covering the entire ACL timeline. In order to disregard non-article publications such as introductions to conference proceedings or letters to the editor, only documents containing abstracts were considered. The corpus was preprocessed using tokenisation, lemmatisation and dependency parsing with the Rasp Parser (Briscoe et al. 2006 ).

Definition of ground truth

Our goal was to define a ground truth for problem and solution strings, while covering as wide a range as possible of syntactic variations in which such strings naturally occur. We also want this ground truth to cover phenomena of problem and solution status which are applicable whether or not the problem or solution status is explicitly mentioned in the text.

To simplify the task, we only consider here problem and solution descriptions that are at most one sentence long. In reality, of course, many problem descriptions and solution descriptions go beyond single sentence, and require for instance an entire paragraph. However, we also know that short summaries of problems and solutions are very prevalent in science, and also that these tend to occur in the most prominent places in a paper. This is because scientists are trained to express their contribution and the obstacles possibly hindering their success, in an informative, succinct manner. That is the reason why we can afford to only look for shorter problem and solution descriptions, ignoring those that cross sentence boundaries.

To define our ground truth, we examined the parsed dependencies and looked for a target word (“problem/solution”) in subject position, and then chose its syntactic argument as our candidate problem or solution phrase. To increase the variation, i.e., to find as many different-worded problem and solution descriptions as possible, we additionally used semantically similar words (near-synonyms) of the target words “problem” or “solution” for the search. Semantic similarity was defined as cosine in a deep learning distributional vector space, trained using Word2Vec (Mikolov et al. 2013 ) on 18,753,472 sentences from a biomedical corpus based on all full-text Pubmed articles (McKeown et al. 2016 ). From the 200 words which were semantically closest to “problem”, we manually selected 28 clear synonyms. These are listed in Table  1 . From the 200 semantically closest words to “solution” we similarly chose 19 (Table  2 ). Of the sentences matching our dependency search, a subset of problem and solution candidate sentences were randomly selected.

Selected words for use in problem candidate phrase extraction

Selected words for use in solution candidate phrase extraction

An example of this is shown in Fig.  2 . Here, the target word “drawback” is in subject position (highlighted in red), and its clausal argument (ccomp) is “(that) it achieves low performance” (highlighted in purple). Examples of other arguments we searched for included copula constructions and direct/indirect objects.

Example of our extraction method for problems using dependencies. (Color figure online)

If more than one candidate was found in a sentence, one was chosen at random. Non-grammatical sentences were excluded; these might appear in the corpus as a result of its source being OCRed text.

800 candidates phrases expressing problems and solutions were automatically extracted (1600 total) and then independently checked for correctness by two annotators (the two authors of this paper). Both authors found the task simple and straightforward. Correctness was defined by two criteria:

• An unexplained phenomenon or a problematic state in science; or
• A research question; or
• An artifact that does not fulfil its stated specification.
• The phrase must not lexically give away its status as problem or solution phrase.

The second criterion saves us from machine learning cues that are too obvious. If for instance, the phrase itself contained the words “lack of” or “problematic” or “drawback”, our manual check rejected it, because it would be too easy for the machine learner to learn such cues, at the expense of many other, more generally occurring cues.

Sampling of negative examples

We next needed to find negative examples for both cases. We wanted them not to stand out on the surface as negative examples, so we chose them so as to mimic the obvious characteristics of the positive examples as closely as possible. We call the negative examples ‘non-problems’ and ‘non-solutions’ respectively. We wanted the only differences between problems and non-problems to be of a semantic nature, nothing that could be read off on the surface. We therefore sampled a population of phrases that obey the same statistical distribution as our problem and solution strings while making sure they really are negative examples. We started from sentences not containing any problem/solution words (i.e. those used as target words). From each such sentence, we at random selected one syntactic subtree contained in it. From these, we randomly selected a subset of negative examples of problems and solutions that satisfy the following conditions:

• The distribution of the head POS tags of the negative strings should perfectly match the head POS tags 3 of the positive strings. This has the purpose of achieving the same proportion of surface syntactic constructions as observed in the positive cases.
• The average lengths of the negative strings must be within a tolerance of the average length of their respective positive candidates e.g., non-solutions must have an average length very similar (i.e. + / -  small tolerance) to solutions. We chose a tolerance value of 3 characters.

Again, a human quality check was performed on non-problems and non-solutions. For each candidate non-problem statement, the candidate was accepted if it did not contain a phenomenon, a problematic state, a research question or a non-functioning artefact. If the string expressed a research task, without explicit statement that there was anything problematic about it (i.e., the ‘wrong’ sense of “problem”, as described above), it was allowed as a non-problem. A clause was confirmed as a non-solution if the string did not represent both a response and positive evaluation.

If the annotator found that the sentence had been slightly mis-parsed, but did contain a candidate, they were allowed to move the boundaries for the candidate clause. This resulted in cleaner text, e.g., in the frequent case of coordination, when non-relevant constituents could be removed.

From the set of sentences which passed the quality-test for both independent assessors, 500 instances of positive and negative problems/solutions were randomly chosen (i.e. 2000 instances in total). When checking for correctness we found that most of the automatically extracted phrases which did not pass the quality test for problem-/solution-hood were either due to obvious learning cues or instances where the sense of problem-hood used is relating to tasks (cf. “ Goal statement and task ” section).

Experimental design

In our experiments, we used three classifiers, namely Naïve Bayes, Logistic Regression and a Support Vector Machine. For all classifiers an implementation from the WEKA machine learning library (Hall et al. 2009 ) was chosen. Given that our dataset is small, tenfold cross-validation was used instead of a held out test set. All significance tests were conducted using the (two-tailed) Sign Test (Siegel 1956 ).

Linguistic correlates of problem- and solution-hood

We first define a set of features without taking the phrase’s context into account. This will tell us about the disambiguation ability of the problem/solution description’s semantics alone. In particular, we cut out the rest of the sentence other than the phrase and never use it for classification. This is done for similar reasons to excluding certain ‘give-away’ phrases inside the phrases themselves (as explained above). As the phrases were found using templates, we know that the machine learner would simply pick up on the semantics of the template, which always contains a synonym of “problem” or “solution”, thus drowning out the more hidden features hopefully inherent in the semantics of the phrases themselves. If we allowed the machine learner to use these stronger features, it would suffer in its ability to generalise to the real task.

ngrams Bags of words are traditionally successfully used for classification tasks in NLP, so we included bags of words (lemmas) within the candidate phrases as one of our features (and treat it as a baseline later on). We also include bigrams and trigrams as multi-word combinations can be indicative of problems and solutions e.g., “combinatorial explosion”.

Polarity Our second feature concerns the polarity of each word in the candidate strings. Consider the following example of a problem taken from our dataset: “very conservative approaches to exact and partial string matches overgenerate badly”. In this sentence, words such as “badly” will be associated with negative polarity, therefore being useful in determining problem-hood. Similarly, solutions will often be associated with a positive sentiment e.g. “smoothing is a good way to overcome data sparsity” . To do this, we perform word sense disambiguation of each word using the Lesk algorithm (Lesk 1986 ). The polarity of the resulting synset in SentiWordNet (Baccianella et al. 2010 ) was then looked up and used as a feature.

Syntax Next, a set of syntactic features were defined by using the presence of POS tags in each candidate. This feature could be helpful in finding syntactic patterns in problems and solutions. We were careful not to base the model directly on the head POS tag and the length of each candidate phrase, as these are defining characteristics used for determining the non-problem and non-solution candidate set.

Negation Negation is an important property that can often greatly affect the polarity of a phrase. For example, a phrase containing a keyword pertinent to solution-hood may be a good indicator but with the presence of negation may flip the polarity to problem-hood e.g., “this can’t work as a solution”. Therefore, presence of negation is determined.

Exemplification and contrast Problems and solutions are often found to be coupled with examples as they allow the author to elucidate their point. For instance, consider the following solution: “Once the translations are generated, an obvious solution is to pick the most fluent alternative, e.g., using an n-gram language model”. (Madnani et al. 2012 ). To acknowledge this, we check for presence of exemplification. In addition to examples, problems in particular are often found when contrast is signalled by the author (e.g. “however, “but”), therefore we also check for presence of contrast in the problem and non-problem candidates only.

Discourse Problems and solutions have also been found to have a correlation with discourse properties. For example, problem-solving patterns often occur in the background sections of a paper. The rationale behind this is that the author is conventionally asked to objectively criticise other work in the background (e.g. describing research gaps which motivate the current paper). To take this in account, we examine the context of each string and capture the section header under which it is contained (e.g. Introduction, Future work). In addition, problems and solutions are often found following the Situation element in the problem-solving pattern (cf. “ Introduction ” section). This preamble setting up the problem or solution means that these elements are likely not to be found occurring at the beginning of a section (i.e. it will usually take some sort of introduction to detail how something is problematic and why a solution is needed). Therefore we record the distance from the candidate string to the nearest section header.

Subcategorisation and adverbials Solutions often involve an activity (e.g. a task). We also model the subcategorisation properties of the verbs involved. Our intuition was that since problematic situations are often described as non-actions, then these are more likely to be intransitive. Conversely solutions are often actions and are likely to have at least one argument. This feature was calculated by running the C&C parser (Curran et al. 2007 ) on each sentence. C&C is a supertagger and parser that has access to subcategorisation information. Solutions are also associated with resultative adverbial modification (e.g. “thus, therefore, consequently”) as it expresses the solutionhood relation between the problem and the solution. It has been seen to occur frequently in problem-solving patterns, as studied by Charles ( 2011 ). Therefore, we check for presence of resultative adverbial modification in the solution and non-solution candidate only.

Embeddings We also wanted to add more information using word embeddings. This was done in two different ways. Firstly, we created a Doc2Vec model (Le and Mikolov 2014 ), which was trained on  ∼  19  million sentences from scientific text (no overlap with our data set). An embedding was created for each candidate sentence. Secondly, word embeddings were calculated using the Word2Vec model (cf. “ Corpus creation ” section). For each candidate head, the full word embedding was included as a feature. Lastly, when creating our polarity feature we query SentiWordNet using synsets assigned by the Lesk algorithm. However, not all words are assigned a sense by Lesk, so we need to take care when that happens. In those cases, the distributional semantic similarity of the word is compared to two words with a known polarity, namely “poor” and “excellent”. These particular words have traditionally been consistently good indicators of polarity status in many studies (Turney 2002 ; Mullen and Collier 2004 ). Semantic similarity was defined as cosine similarity on the embeddings of the Word2Vec model (cf. “ Corpus creation ” section).

Modality Responses to problems in scientific writing often express possibility and necessity, and so have a close connection with modality. Modality can be broken into three main categories, as described by Kratzer ( 1991 ), namely epistemic (possibility), deontic (permission / request / wish) and dynamic (expressing ability).

Problems have a strong relationship to modality within scientific writing. Often, this is due to a tactic called “hedging” (Medlock and Briscoe 2007 ) where the author uses speculative language, often using Epistemic modality, in an attempt to make either noncommital or vague statements. This has the effect of allowing the author to distance themselves from the statement, and is often employed when discussing negative or problematic topics. Consider the following example of Epistemic modality from Nakov and Hearst ( 2008 ): “A potential drawback is that it might not work well for low-frequency words”.

To take this linguistic correlate into account as a feature, we replicated a modality classifier as described by (Ruppenhofer and Rehbein 2012 ). More sophisticated modality classifiers have been recently introduced, for instance using a wide range of features and convolutional neural networks, e.g, (Zhou et al. 2015 ; Marasović and Frank 2016 ). However, we wanted to check the effect of a simpler method of modality classification on the final outcome first before investing heavily into their implementation. We trained three classifiers using the subset of features which Ruppenhofer et al. reported as performing best, and evaluated them on the gold standard dataset provided by the authors 4 . The results of the are shown in Table  3 . The dataset contains annotations of English modal verbs on the 535 documents of the first MPQA corpus release (Wiebe et al. 2005 ).

Modality classifier results (precision/recall/f-measure) using Naïve Bayes (NB), logistic regression, and a support vector machine (SVM)

Italicized results reflect highest f-measure reported per modal category

Logistic Regression performed best overall and so this model was chosen for our upcoming experiments. With regards to the optative and concessive modal categories, they can be seen to perform extremely poorly, with the optative category receiving a null score across all three classifiers. This is due to a limitation in the dataset, which is unbalanced and contains very few instances of these two categories. This unbalanced data also is the reason behind our decision of reporting results in terms of recall, precision and f-measure in Table  3 .

The modality classifier was then retrained on the entirety of the dataset used by Ruppenhofer and Rehbein ( 2012 ) using the best performing model from training (Logistic Regression). This new model was then used in the upcoming experiment to predict modality labels for each instance in our dataset.

As can be seen from Table  4 , we are able to achieve good results for distinguishing a problematic statement from non-problematic one. The bag-of-words baseline achieves a very good performance of 71.0% for the Logistic Regression classifier, showing that there is enough signal in the candidate phrases alone to distinguish them much better than random chance.

Results distinguishing problems from non-problems using Naïve Bayes (NB), logistic regression (LR) and a support vector machine (SVM)

Each feature set’s performance is shown in isolation followed by combinations with other features. Tenfold stratified cross-validation was used across all experiments. Statistical significance with respect to the baseline at the p  < 0.05 , 0.01, 0.001 levels is denoted by *, ** and *** respectively

Taking a look at Table  5 , which shows the information gain for the top lemmas,

Information gain (IG) in bits of top lemmas from the bag-of-words baseline in Table  4

we can see that the top lemmas are indeed indicative of problemhood (e.g. “limit”,“explosion”). Bigrams achieved good performance on their own (as did negation and discourse) but unfortunately performance deteriorated when using trigrams, particularly with the SVM and LR. The subcategorisation feature was the worst performing feature in isolation. Upon taking a closer look at our data, we saw that our hypothesis that intransitive verbs are commonly used in problematic statements was true, with over 30% of our problems (153) using them. However, due to our sampling method for the negative cases we also picked up many intransitive verbs (163). This explains the almost random chance performance (i.e.  50%) given that the distribution of intransitive verbs amongst the positive and negative candidates was almost even.

The modality feature was the most expensive to produce, but also didn’t perform very well is isolation. This surprising result may be partly due to a data sparsity issue

where only a small portion (169) of our instances contained modal verbs. The breakdown of how many types of modal senses which occurred is displayed in Table  6 . The most dominant modal sense was epistemic. This is a good indicator of problemhood (e.g. hedging, cf. “ Linguistic correlates of problem- and solution-hood ” section) but if the accumulation of additional data was possible, we think that this feature may have the potential to be much more valuable in determining problemhood. Another reason for the performance may be domain dependence of the classifier since it was trained on text from different domains (e.g. news). Additionally, modality has also shown to be helpful in determining contextual polarity (Wilson et al. 2005 ) and argumentation (Becker et al. 2016 ), so using the output from this modality classifier may also prove useful for further feature engineering taking this into account in future work.

Number of instances of modal senses

Polarity managed to perform well but not as good as we hoped. However, this feature also suffers from a sparsity issue resulting from cases where the Lesk algorithm (Lesk 1986 ) is not able to resolve the synset of the syntactic head.

Knowledge of syntax provides a big improvement with a significant increase over the baseline results from two of the classifiers.

Examining this in greater detail, POS tags with high information gain mostly included tags from open classes (i.e. VB-, JJ-, NN- and RB-). These tags are often more associated with determining polarity status than tags such as prepositions and conjunctions (i.e. adverbs and adjectives are more likely to be describing something with a non-neutral viewpoint).

The embeddings from Doc2Vec allowed us to obtain another significant increase in performance (72.9% with Naïve Bayes) over the baseline and polarity using Word2Vec provided the best individual feature result (77.2% with SVM).

Combining all features together, each classifier managed to achieve a significant result over the baseline with the best result coming from the SVM (81.8%). Problems were also better classified than non-problems as shown in the confusion matrix in Table  7 . The addition of the Word2Vec vectors may be seen as a form of smoothing in cases where previous linguistic features had a sparsity issue i.e., instead of a NULL entry, the embeddings provide some sort of value for each candidate. Particularly wrt. the polarity feature, cases where Lesk was unable to resolve a synset meant that a ZERO entry was added to the vector supplied to the machine learner. Amongst the possible combinations, the best subset of features was found by combining all features with the exception of bigrams, trigrams, subcategorisation and modality. This subset of features managed to improve results in both the Naïve Bayes and SVM classifiers with the highest overall result coming from the SVM (82.3%).

Confusion matrix for problems

The results for disambiguation of solutions from non-solutions can be seen in Table  8 . The bag-of-words baseline performs much better than random, with the performance being quite high with regard to the SVM (this result was also higher than any of the baseline performances from the problem classifiers). As shown in Table  9 , the top ranked lemmas from the best performing model (using information gain) included “use” and “method”. These lemmas are very indicative of solutionhood and so give some insight into the high baseline returned from the machine learners. Subcategorisation and the result adverbials were the two worst performing features. However, the low performance for subcategorisation is due to the sampling of the non-solutions (the same reason for the low performance of the problem transitivity feature). When fitting the POS-tag distribution for the negative samples, we noticed that over 80% of the head POS-tags were verbs (much higher than the problem heads). The most frequent verb type being the infinite form.

Results distinguishing solutions from non-solutions using Naïve Bayes (NB), logistic regression (LR) and a support vector machine (SVM)

Each feature set’s performance is shown in isolation followed by combinations with other features. Tenfold stratified cross-validation was used across all experiments

Information gain (IG) in bits of top lemmas from the bag-of-words baseline in Table  8

This is not surprising given that a very common formulation to describe a solution is to use the infinitive “TO” since it often describes a task e.g., “One solution is to find the singletons and remove them”. Therefore, since the head POS tags of the non-solutions had to match this high distribution of infinitive verbs present in the solution, the subcategorisation feature is not particularly discriminatory. Polarity, negation, exemplification and syntactic features were slightly more discriminate and provided comparable results. However, similar to the problem experiment, the embeddings from Word2Vec and Doc2Vec proved to be the best features, with polarity using Word2Vec providing the best individual result (73.4% with SVM).

Combining all features together managed to improve over each feature in isolation and beat the baseline using all three classifiers. Furthermore, when looking at the confusion matrix in Table  10 the solutions were classified more accurately than the non-solutions. The best subset of features was found by combining all features without adverbial of result, bigrams, exemplification, negation, polarity and subcategorisation. The best result using this subset of features was achieved by the SVM with 79.7%. It managed to greatly improve upon the baseline but was just shy of achieving statistical significance ( p = 0.057 ).

Confusion matrix for solutions

In this work, we have presented new supervised classifiers for the task of identifying problem and solution statements in scientific text. We have also introduced a new corpus for this task and used it for evaluating our classifiers. Great care was taken in constructing the corpus by ensuring that the negative and positive samples were closely matched in terms of syntactic shape. If we had simply selected random subtrees for negative samples without regard for any syntactic similarity with our positive samples, the machine learner may have found easy signals such as sentence length. Additionally, since we did not allow the machine learner to see the surroundings of the candidate string within the sentence, this made our task even harder. Our performance on the corpus shows promise for this task, and proves that there are strong signals for determining both the problem and solution parts of the problem-solving pattern independently.

With regard to classifying problems from non-problems, features such as the POS tag, document and word embeddings provide the best features, with polarity using the Word2Vec embeddings achieving the highest feature performance. The best overall result was achieved using an SVM with a subset of features (82.3%). Classifying solutions from non-solutions also performs well using the embedding features, with the best feature also being polarity using the Word2Vec embeddings, and the highest result also coming from the SVM with a feature subset (79.7%).

In future work, we plan to link problem and solution statements which were found independently during our corpus creation. Given that our classifiers were trained on data solely from the ACL anthology, we also hope to investigate the domain specificity of our classifiers and see how well they can generalise to domains other than ACL (e.g. bioinformatics). Since we took great care at removing the knowledge our classifiers have of the explicit statements of problem and solution (i.e. the classifiers were trained only on the syntactic argument of the explicit statement of problem-/solution-hood), our classifiers should in principle be in a good position to generalise, i.e., find implicit statements too. In future work, we will measure to which degree this is the case.

To facilitate further research on this topic, all code and data used in our experiments can be found here: www.cl.cam.ac.uk/~kh562/identifying-problems-and-solutions.html

Acknowledgements

The first author has been supported by an EPSRC studentship (Award Ref: 1641528). We thank the reviewers for their helpful comments.

1 http://acl-arc.comp.nus.edu.sg/ .

2 The corpus comprises 3,391,198 sentences, 71,149,169 words and 451,996,332 characters.

3 The head POS tags were found using a modification of the Collins’ Head Finder. This modified algorithm addresses some of the limitations of the head finding heuristics described by Collins ( 2003 ) and can be found here: http://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/trees/ModCollinsHeadFinder.html .

4 https://www.uni-hildesheim.de/ruppenhofer/data/modalia_release1.0.tgz.

Contributor Information

Kevin Heffernan, Email: [email protected] .

Simone Teufel, Email: [email protected] .

• Baccianella S, Esuli A, Sebastiani F. Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. LREC. 2010; 10 :2200–2204. [ Google Scholar ]
• Becker, M., Palmer, A., & Frank, A. (2016). Clause types and modality in argumentative microtexts. In Workshop on foundations of the language of argumentation (in conjunction with COMMA) .
• Briscoe, T., Carroll, J., & Watson, R. (2006). The second release of the rasp system. In Proceedings of the COLING/ACL on interactive presentation sessions, association for computational linguistics pp. 77–80.
• Chandrasekaran B. Towards a taxonomy of problem solving types. AI Magazine. 1983; 4 (1):9. [ Google Scholar ]
• Charles M. Adverbials of result: Phraseology and functions in the problem-solution pattern. Journal of English for Academic Purposes. 2011; 10 (1):47–60. doi: 10.1016/j.jeap.2011.01.002. [ CrossRef ] [ Google Scholar ]
• Chen, D., Dyer, C., Cohen, S. B., & Smith, N. A. (2011). Unsupervised bilingual pos tagging with markov random fields. In Proceedings of the first workshop on unsupervised learning in NLP, association for computational linguistics pp. 64–71.
• Collins M. Head-driven statistical models for natural language parsing. Computational Linguistics. 2003; 29 (4):589–637. doi: 10.1162/089120103322753356. [ CrossRef ] [ Google Scholar ]
• Councill, I. G., Giles, C. L., & Kan, M. Y. (2008). Parscit: An open-source CRF reference string parsing package. In LREC .
• Curran, J. R., Clark, S., & Bos, J. (2007). Linguistically motivated large-scale NLP with C&C and boxer. In Proceedings of the 45th annual meeting of the ACL on interactive poster and demonstration sessions, association for computational linguistics pp. 33–36.
• Flowerdew L. Corpus-based analyses of the problem-solution pattern: A phraseological approach. Amsterdam: John Benjamins Publishing; 2008. [ Google Scholar ]
• Grimes JE. The thread of discourse. Berlin: Walter de Gruyter; 1975. [ Google Scholar ]
• Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH. The weka data mining software: An update. ACM SIGKDD Explorations Newsletter. 2009; 11 (1):10–18. doi: 10.1145/1656274.1656278. [ CrossRef ] [ Google Scholar ]
• Hoey M. Textual interaction: An introduction to written discourse analysis. Portland: Psychology Press; 2001. [ Google Scholar ]
• Hutchins J. On the structure of scientific texts. UEA Papers in Linguistics. 1977; 5 (3):18–39. [ Google Scholar ]
• Jonassen DH. Toward a design theory of problem solving. Educational Technology Research and Development. 2000; 48 (4):63–85. doi: 10.1007/BF02300500. [ CrossRef ] [ Google Scholar ]
• Jordan MP. Short texts to explain problem-solution structures-and vice versa. Instructional Science. 1980; 9 (3):221–252. doi: 10.1007/BF00177328. [ CrossRef ] [ Google Scholar ]
• Kratzer, A. (1991). Modality. In von Stechow & Wunderlich (Eds.), Semantics: An international handbook of contemporary research .
• Le QV, Mikolov T. Distributed representations of sentences and documents. ICML. 2014; 14 :1188–1196. [ Google Scholar ]
• Lesk, M. (1986). Automatic sense disambiguation using machine readable dictionaries: How to tell a pine cone from an ice cream cone. In Proceedings of the 5th annual international conference on Systems documentation, ACM (pp. 24–26).
• Madnani, N., Tetreault, J., & Chodorow, M. (2012). Exploring grammatical error correction with not-so-crummy machine translation. In Proceedings of the seventh workshop on building educational applications using NLP, association for computational linguistics pp. 44–53.
• Mann WC, Thompson SA. Rhetorical structure theory: Toward a functional theory of text organization. Text-Interdisciplinary Journal for the Study of Discourse. 1988; 8 (3):243–281. doi: 10.1515/text.1.1988.8.3.243. [ CrossRef ] [ Google Scholar ]
• Marasović, A., & Frank, A. (2016). Multilingual modal sense classification using a convolutional neural network. In Proceedings of the 1st Workshop on Representation Learning for NLP .
• McKeown K, Daume H, Chaturvedi S, Paparrizos J, Thadani K, Barrio P, Biran O, Bothe S, Collins M, Fleischmann KR, et al. Predicting the impact of scientific concepts using full-text features. Journal of the Association for Information Science and Technology. 2016; 67 :2684–2696. doi: 10.1002/asi.23612. [ CrossRef ] [ Google Scholar ]
• Medlock B, Briscoe T. Weakly supervised learning for hedge classification in scientific literature. ACL, Citeseer. 2007; 2007 :992–999. [ Google Scholar ]
• Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (pp. 3111–3119).
• Mohit, B., Schneider, N., Bhowmick, R., Oflazer, K., & Smith, N. A. (2012). Recall-oriented learning of named entities in arabic wikipedia. In Proceedings of the 13th conference of the European chapter of the association for computational linguistics, association for computational linguistics (pp. 162–173).
• Mullen T, Collier N. Sentiment analysis using support vector machines with diverse information sources. EMNLP. 2004; 4 :412–418. [ Google Scholar ]
• Nakov, P., Hearst, M. A. (2008). Solving relational similarity problems using the web as a corpus. In: ACL (pp. 452–460).
• Poon, H., & Domingos, P. (2009). Unsupervised semantic parsing. In Proceedings of the 2009 conference on empirical methods in natural language processing: Volume 1-association for computational linguistics (pp. 1–10).
• Ruppenhofer, J., & Rehbein, I. (2012). Yes we can!? Annotating the senses of English modal verbs. In Proceedings of the 8th international conference on language resources and evaluation (LREC), Citeseer (pp. 24–26).
• Saha, S. K., Mitra, P., & Sarkar, S. (2008). Word clustering and word selection based feature reduction for maxent based hindi ner. In ACL (pp. 488–495).
• Scott, M. (2001). Mapping key words to problem and solution. In Patterns of text: In honour of Michael Hoey Benjamins, Amsterdam (pp. 109–127).
• Siegel S. Nonparametric statistics for the behavioral sciences. New York: McGraw-hill; 1956. [ Google Scholar ]
• Strübing, J. (2007). Research as pragmatic problem-solving: The pragmatist roots of empirically-grounded theorizing. In The Sage handbook of grounded theory (pp. 580–602).
• Teufel, S., et al. (2000). Argumentative zoning: Information extraction from scientific text. PhD Thesis, Citeseer .
• Turney, P. D. (2002). Thumbs up or thumbs down?: Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th annual meeting on association for computational linguistics, association for computational linguistics (pp. 417–424).
• Van Dijk TA. Text and context explorations in the semantics and pragmatics of discourse. London: Longman; 1980. [ Google Scholar ]
• Wiebe J, Wilson T, Cardie C. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation. 2005; 39 (2):165–210. doi: 10.1007/s10579-005-7880-9. [ CrossRef ] [ Google Scholar ]
• Wilson, T., Wiebe, J., & Hoffmann, P. (2005). Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the conference on human language technology and empirical methods in natural language processing, association for computational linguistics (pp. 347–354).
• Winter, E. O. (1968). Some aspects of cohesion. In Sentence and clause in scientific English . University College London.
• Zhou, M., Frank, A., Friedrich, A., & Palmer, A. (2015). Semantically enriched models for modal sense classification. In Workshop on linking models of lexical, sentential and discourse-level semantics (LSDSem) (p. 44).

Change Password

Your password must have 8 characters or more and contain 3 of the following:.

• a lower case character, 
• an upper case character, 
• a special character 

Password Changed Successfully

Your password has been changed

• Sign in / Register

Request Username

Can't sign in? Forgot your username?

Enter your email address below and we will send you your username

If the address matches an existing account you will receive an email with instructions to retrieve your username

Teaching Creativity and Inventive Problem Solving in Science

• Robert L. DeHaan

Division of Educational Studies, Emory University, Atlanta, GA 30322

Search for more papers by this author

Engaging learners in the excitement of science, helping them discover the value of evidence-based reasoning and higher-order cognitive skills, and teaching them to become creative problem solvers have long been goals of science education reformers. But the means to achieve these goals, especially methods to promote creative thinking in scientific problem solving, have not become widely known or used. In this essay, I review the evidence that creativity is not a single hard-to-measure property. The creative process can be explained by reference to increasingly well-understood cognitive skills such as cognitive flexibility and inhibitory control that are widely distributed in the population. I explore the relationship between creativity and the higher-order cognitive skills, review assessment methods, and describe several instructional strategies for enhancing creative problem solving in the college classroom. Evidence suggests that instruction to support the development of creativity requires inquiry-based teaching that includes explicit strategies to promote cognitive flexibility. Students need to be repeatedly reminded and shown how to be creative, to integrate material across subject areas, to question their own assumptions, and to imagine other viewpoints and possibilities. Further research is required to determine whether college students' learning will be enhanced by these measures.

INTRODUCTION

Dr. Dunne paces in front of his section of first-year college students, today not as their Bio 110 teacher but in the role of facilitator in their monthly “invention session.” For this meeting, the topic is stem cell therapy in heart disease. Members of each team of four students have primed themselves on the topic by reading selected articles from accessible sources such as Science, Nature, and Scientific American, and searching the World Wide Web, triangulating for up-to-date, accurate, background information. Each team knows that their first goal is to define a set of problems or limitations to overcome within the topic and to begin to think of possible solutions. Dr. Dunne starts the conversation by reminding the group of the few ground rules: one speaker at a time, listen carefully and have respect for others' ideas, question your own and others' assumptions, focus on alternative paths or solutions, maintain an atmosphere of collaboration and mutual support. He then sparks the discussion by asking one of the teams to describe a problem in need of solution.

Science in the United States is widely credited as a major source of discovery and economic development. According to the 2005 TAP Report produced by a prominent group of corporate leaders, “To maintain our country's competitiveness in the twenty-first century, we must cultivate the skilled scientists and engineers needed to create tomorrow's innovations.” ( www.tap2015.org/about/TAP_report2.pdf ). A panel of scientists, engineers, educators, and policy makers convened by the National Research Council (NRC) concurred with this view, reporting that the vitality of the nation “is derived in large part from the productivity of well-trained people and the steady stream of scientific and technical innovations they produce” ( NRC, 2007 ).

For many decades, science education reformers have promoted the idea that learners should be engaged in the excitement of science; they should be helped to discover the value of evidence-based reasoning and higher-order cognitive skills, and be taught to become innovative problem solvers (for reviews, see DeHaan, 2005 ; Hake, 2005 ; Nelson, 2008 ; Perkins and Wieman, 2008 ). But the means to achieve these goals, especially methods to promote creative thinking in scientific problem solving, are not widely known or used. An invention session such as that led by the fictional Dr. Dunne, described above, may seem fanciful as a means of teaching students to think about science as something more than a body of facts and terms to memorize. In recent years, however, models for promoting creative problem solving were developed for classroom use, as detailed by Treffinger and Isaksen (2005) , and such techniques are often used in the real world of high technology. To promote imaginative thinking, the advertising executive Alex F. Osborn invented brainstorming ( Osborn, 1948 , 1979 ), a technique that has since been successful in stimulating inventiveness among engineers and scientists. Could such strategies be transferred to a class for college students? Could they serve as a supplement to a high-quality, scientific teaching curriculum that helps students learn the facts and conceptual frameworks of science and make progress along the novice–expert continuum? Could brainstorming or other instructional strategies that are specifically designed to promote creativity teach students to be more adaptive in their growing expertise, more innovative in their problem-solving abilities? To begin to answer those questions, we first need to understand what is meant by “creativity.”

What Is Creativity? Big-C versus Mini-C Creativity

How to define creativity is an age-old question. Justice Potter Stewart's famous dictum regarding obscenity “I know it when I see it” has also long been an accepted test of creativity. But this is not an adequate criterion for developing an instructional approach. A scientist colleague of mine recently noted that “Many of us [in the scientific community] rarely give the creative process a second thought, imagining one either ‘has it’ or doesn't.” We often think of inventiveness or creativity in scientific fields as the kind of gift associated with a Michelangelo or Einstein. This is what Kaufman and Beghetto (2008) call big-C creativity, borrowing the term that earlier workers applied to the talents of experts in various fields who were identified as particularly creative by their expert colleagues ( MacKinnon, 1978 ). In this sense, creativity is seen as the ability of individuals to generate new ideas that contribute substantially to an intellectual domain. Howard Gardner defined such a creative person as one who “regularly solves problems, fashions products, or defines new questions in a domain in a way that is initially considered novel but that ultimately comes to be accepted in a particular cultural setting” ( Gardner, 1993 , p. 35).

But there is another level of inventiveness termed by various authors as “little-c” ( Craft, 2000 ) or “mini-c” ( Kaufman and Beghetto, 2008 ) creativity that is widespread among all populations. This would be consistent with the workplace definition of creativity offered by Amabile and her coworkers: “coming up with fresh ideas for changing products, services and processes so as to better achieve the organization's goals” ( Amabile et al. , 2005 ). Mini-c creativity is based on what Craft calls “possibility thinking” ( Craft, 2000 , pp. 3–4), as experienced when a worker suddenly has the insight to visualize a new, improved way to accomplish a task; it is represented by the “aha” moment when a student first sees two previously disparate concepts or facts in a new relationship, an example of what Arthur Koestler identified as bisociation: “perceiving a situation or event in two habitually incompatible associative contexts” ( Koestler, 1964 , p. 95).

In this essay, I maintain that mini-c creativity is not a mysterious, innate endowment of rare individuals. Instead, I argue that creative thinking is a multicomponent process, mediated through social interactions, that can be explained by reference to increasingly well-understood mental abilities such as cognitive flexibility and cognitive control that are widely distributed in the population. Moreover, I explore some of the recent research evidence (though with no effort at a comprehensive literature review) showing that these mental abilities are teachable; like other higher-order cognitive skills (HOCS), they can be enhanced by explicit instruction.

Creativity Is a Multicomponent Process

Efforts to define creativity in psychological terms go back to J. P. Guilford ( Guilford, 1950 ) and E. P. Torrance ( Torrance, 1974 ), both of whom recognized that underlying the construct were other cognitive variables such as ideational fluency, originality of ideas, and sensitivity to missing elements. Many authors since then have extended the argument that a creative act is not a singular event but a process, an interplay among several interactive cognitive and affective elements. In this view, the creative act has two phases, a generative and an exploratory or evaluative phase ( Finke et al. , 1996 ). During the generative process, the creative mind pictures a set of novel mental models as potential solutions to a problem. In the exploratory phase, we evaluate the multiple options and select the best one. Early scholars of creativity, such as J. P. Guilford, characterized the two phases as divergent thinking and convergent thinking ( Guilford, 1950 ). Guilford defined divergent thinking as the ability to produce a broad range of associations to a given stimulus or to arrive at many solutions to a problem (for overviews of the field from different perspectives, see Amabile, 1996 ; Banaji et al. , 2006 ; Sawyer, 2006 ). In neurocognitive terms, divergent thinking is referred to as associative richness ( Gabora, 2002 ; Simonton, 2004 ), which is often measured experimentally by comparing the number of words that an individual generates from memory in response to stimulus words on a word association test. In contrast, convergent thinking refers to the capacity to quickly focus on the one best solution to a problem.

The idea that there are two stages to the creative process is consistent with results from cognition research indicating that there are two distinct modes of thought, associative and analytical ( Neisser, 1963 ; Sloman, 1996 ). In the associative mode, thinking is defocused, suggestive, and intuitive, revealing remote or subtle connections between items that may be correlated, or may not, and are usually not causally related ( Burton, 2008 ). In the analytical mode, thought is focused and evaluative, more conducive to analyzing relationships of cause and effect (for a review of other cognitive aspects of creativity, see Runco, 2004 ). Science educators associate the analytical mode with the upper levels (analysis, synthesis, and evaluation) of Bloom's taxonomy (e.g., Crowe et al. , 2008 ), or with “critical thinking,” the process that underlies the “purposeful, self-regulatory judgment that drives problem-solving and decision-making” ( Quitadamo et al. , 2008 , p. 328). These modes of thinking are under cognitive control through the executive functions of the brain. The core executive functions, which are thought to underlie all planning, problem solving, and reasoning, are defined ( Blair and Razza, 2007 ) as working memory control (mentally holding and retrieving information), cognitive flexibility (considering multiple ideas and seeing different perspectives), and inhibitory control (resisting several thoughts or actions to focus on one). Readers wishing to delve further into the neuroscience of the creative process can refer to the cerebrocerebellar theory of creativity ( Vandervert et al. , 2007 ) in which these mental activities are described neurophysiologically as arising through interactions among different parts of the brain.

The main point from all of these works is that creativity is not some single hard-to-measure property or act. There is ample evidence that the creative process requires both divergent and convergent thinking and that it can be explained by reference to increasingly well-understood underlying mental abilities ( Haring-Smith, 2006 ; Kim, 2006 ; Sawyer, 2006 ; Kaufman and Sternberg, 2007 ) and cognitive processes ( Simonton, 2004 ; Diamond et al. , 2007 ; Vandervert et al. , 2007 ).

Creativity Is Widely Distributed and Occurs in a Social Context

Although it is understandable to speak of an aha moment as a creative act by the person who experiences it, authorities in the field have long recognized (e.g., Simonton, 1975 ) that creative thinking is not so much an individual trait but rather a social phenomenon involving interactions among people within their specific group or cultural settings. “Creativity isn't just a property of individuals, it is also a property of social groups” ( Sawyer, 2006 , p. 305). Indeed, Osborn introduced his brainstorming method because he was convinced that group creativity is always superior to individual creativity. He drew evidence for this conclusion from activities that demand collaborative output, for example, the improvisations of a jazz ensemble. Although each musician is individually creative during a performance, the novelty and inventiveness of each performer's playing is clearly influenced, and often enhanced, by “social and interactional processes” among the musicians ( Sawyer, 2006 , p. 120). Recently, Brophy (2006) offered evidence that for problem solving, the situation may be more nuanced. He confirmed that groups of interacting individuals were better at solving complex, multipart problems than single individuals. However, when dealing with certain kinds of single-issue problems, individual problem solvers produced a greater number of solutions than interacting groups, and those solutions were judged to be more original and useful.

Consistent with the findings of Brophy (2006) , many scholars acknowledge that creative discoveries in the real world such as solving the problems of cutting-edge science—which are usually complex and multipart—are influenced or even stimulated by social interaction among experts. The common image of the lone scientist in the laboratory experiencing a flash of creative inspiration is probably a myth from earlier days. As a case in point, the science historian Mara Beller analyzed the social processes that underlay some of the major discoveries of early twentieth-century quantum physics. Close examination of successive drafts of publications by members of the Copenhagen group revealed a remarkable degree of influence and collaboration among 10 or more colleagues, although many of these papers were published under the name of a single author ( Beller, 1999 ). Sociologists Bruno Latour and Steve Woolgar's study ( Latour and Woolgar, 1986 ) of a neuroendocrinology laboratory at the Salk Institute for Biological Studies make the related point that social interactions among the participating scientists determined to a remarkable degree what discoveries were made and how they were interpreted. In the laboratory, researchers studied the chemical structure of substances released by the brain. By analysis of the Salk scientists' verbalizations of concepts, theories, formulas, and results of their investigations, Latour and Woolgar showed that the structures and interpretations that were agreed upon, that is, the discoveries announced by the laboratory, were mediated by social interactions and power relationships among members of the laboratory group. By studying the discovery process in other fields of the natural sciences, sociologists and anthropologists have provided more cases that further illustrate how social and cultural dimensions affect scientific insights (for a thoughtful review, see Knorr Cetina, 1995 ).

In sum, when an individual experiences an aha moment that feels like a singular creative act, it may rather have resulted from a multicomponent process, under the influence of group interactions and social context. The process that led up to what may be sensed as a sudden insight will probably have included at least three diverse, but testable elements: 1) divergent thinking, including ideational fluency or cognitive flexibility, which is the cognitive executive function that underlies the ability to visualize and accept many ideas related to a problem; 2) convergent thinking or the application of inhibitory control to focus and mentally evaluate ideas; and 3) analogical thinking, the ability to understand a novel idea in terms of one that is already familiar.

LITERATURE REVIEW

What do we know about how to teach creativity.

The possibility of teaching for creative problem solving gained credence in the 1960s with the studies of Jerome Bruner, who argued that children should be encouraged to “treat a task as a problem for which one invents an answer, rather than finding one out there in a book or on the blackboard” ( Bruner, 1965 , pp. 1013–1014). Since that time, educators and psychologists have devised programs of instruction designed to promote creativity and inventiveness in virtually every student population: pre–K, elementary, high school, and college, as well as in disadvantaged students, athletes, and students in a variety of specific disciplines (for review, see Scott et al. , 2004 ). Smith (1998) identified 172 instructional approaches that have been applied at one time or another to develop divergent thinking skills.

Some of the most convincing evidence that elements of creativity can be enhanced by instruction comes from work with young children. Bodrova and Leong (2001) developed the Tools of the Mind (Tools) curriculum to improve all of the three core mental executive functions involved in creative problem solving: cognitive flexibility, working memory, and inhibitory control. In a year-long randomized study of 5-yr-olds from low-income families in 21 preschool classrooms, half of the teachers applied the districts' balanced literacy curriculum (literacy), whereas the experimenters trained the other half to teach the same academic content by using the Tools curriculum ( Diamond et al. , 2007 ). At the end of the year, when the children were tested with a battery of neurocognitive tests including a test for cognitive flexibility ( Durston et al. , 2003 ; Davidson et al. , 2006 ), those exposed to the Tools curriculum outperformed the literacy children by as much as 25% ( Diamond et al. , 2007 ). Although the Tools curriculum and literacy program were similar in academic content and in many other ways, they differed primarily in that Tools teachers spent 80% of their time explicitly reminding the children to think of alternative ways to solve a problem and building their executive function skills.

Teaching older students to be innovative also demands instruction that explicitly promotes creativity but is rigorously content-rich as well. A large body of research on the differences between novice and expert cognition indicates that creative thinking requires at least a minimal level of expertise and fluency within a knowledge domain ( Bransford et al. , 2000 ; Crawford and Brophy, 2006 ). What distinguishes experts from novices, in addition to their deeper knowledge of the subject, is their recognition of patterns in information, their ability to see relationships among disparate facts and concepts, and their capacity for organizing content into conceptual frameworks or schemata ( Bransford et al. , 2000 ; Sawyer, 2005 ).

Such expertise is often lacking in the traditional classroom. For students attempting to grapple with new subject matter, many kinds of problems that are presented in high school or college courses or that arise in the real world can be solved merely by applying newly learned algorithms or procedural knowledge. With practice, problem solving of this kind can become routine and is often considered to represent mastery of a subject, producing what Sternberg refers to as “pseudoexperts” ( Sternberg, 2003 ). But beyond such routine use of content knowledge the instructor's goal must be to produce students who have gained the HOCS needed to apply, analyze, synthesize, and evaluate knowledge ( Crowe et al. , 2008 ). The aim is to produce students who know enough about a field to grasp meaningful patterns of information, who can readily retrieve relevant knowledge from memory, and who can apply such knowledge effectively to novel problems. This condition is referred to as adaptive expertise ( Hatano and Ouro, 2003 ; Schwartz et al. , 2005 ). Instead of applying already mastered procedures, adaptive experts are able to draw on their knowledge to invent or adapt strategies for solving unique or novel problems within a knowledge domain. They are also able, ideally, to transfer conceptual frameworks and schemata from one domain to another (e.g., Schwartz et al. , 2005 ). Such flexible, innovative application of knowledge is what results in inventive or creative solutions to problems ( Crawford and Brophy, 2006 ; Crawford, 2007 ).

Promoting Creative Problem Solving in the College Classroom

In most college courses, instructors teach science primarily through lectures and textbooks that are dominated by facts and algorithmic processing rather than by concepts, principles, and evidence-based ways of thinking. This is despite ample evidence that many students gain little new knowledge from traditional lectures ( Hrepic et al. , 2007 ). Moreover, it is well documented that these methods engender passive learning rather than active engagement, boredom instead of intellectual excitement, and linear thinking rather than cognitive flexibility (e.g., Halpern and Hakel, 2003 ; Nelson, 2008 ; Perkins and Wieman, 2008 ). Cognitive flexibility, as noted, is one of the three core mental executive functions involved in creative problem solving ( Ausubel, 1963 , 2000 ). The capacity to apply ideas creatively in new contexts, referred to as the ability to “transfer” knowledge (see Mestre, 2005 ), requires that learners have opportunities to actively develop their own representations of information to convert it to a usable form. Especially when a knowledge domain is complex and fraught with ill-structured information, as in a typical introductory college biology course, instruction that emphasizes active-learning strategies is demonstrably more effective than traditional linear teaching in reducing failure rates and in promoting learning and transfer (e.g., Freeman et al. , 2007 ). Furthermore, there is already some evidence that inclusion of creativity training as part of a college curriculum can have positive effects. Hunsaker (2005) has reviewed a number of such studies. He cites work by McGregor (2001) , for example, showing that various creativity training programs including brainstorming and creative problem solving increase student scores on tests of creative-thinking abilities.

Model creativity—students develop creativity when instructors model creative thinking and inventiveness.

Repeatedly encourage idea generation—students need to be reminded to generate their own ideas and solutions in an environment free of criticism.

Cross-fertilize ideas—where possible, avoid teaching in subject-area boxes: a math box, a social studies box, etc; students' creative ideas and insights often result from learning to integrate material across subject areas.

Build self-efficacy—all students have the capacity to create and to experience the joy of having new ideas, but they must be helped to believe in their own capacity to be creative.

Constantly question assumptions—make questioning a part of the daily classroom exchange; it is more important for students to learn what questions to ask and how to ask them than to learn the answers.

Imagine other viewpoints—students broaden their perspectives by learning to reflect upon ideas and concepts from different points of view.

How Is Creativity Related to Critical Thinking and the Higher-Order Cognitive Skills?

It is not uncommon to associate creativity and ingenuity with scientific reasoning ( Sawyer, 2005 ; 2006 ). When instructors apply scientific teaching strategies ( Handelsman et al. , 2004 ; DeHaan, 2005 ; Wood, 2009 ) by using instructional methods based on learning research, according to Ebert-May and Hodder ( 2008 ), “we see students actively engaged in the thinking, creativity, rigor, and experimentation we associate with the practice of science—in much the same way we see students learn in the field and in laboratories” (p. 2). Perkins and Wieman (2008) note that “To be successful innovators in science and engineering, students must develop a deep conceptual understanding of the underlying science ideas, an ability to apply these ideas and concepts broadly in different contexts, and a vision to see their relevance and usefulness in real-world applications … An innovator is able to perceive and realize potential connections and opportunities better than others” (pp. 181–182). The results of Scott et al. (2004) suggest that nontraditional courses in science that are based on constructivist principles and that use strategies of scientific teaching to promote the HOCS and enhance content mastery and dexterity in scientific thinking ( Handelsman et al. , 2007 ; Nelson, 2008 ) also should be effective in promoting creativity and cognitive flexibility if students are explicitly guided to learn these skills.

Creativity is an essential element of problem solving ( Mumford et al. , 1991 ; Runco, 2004 ) and of critical thinking ( Abrami et al. , 2008 ). As such, it is common to think of applications of creativity such as inventiveness and ingenuity among the HOCS as defined in Bloom's taxonomy ( Crowe et al. , 2008 ). Thus, it should come as no surprise that creativity, like other elements of the HOCS, can be taught most effectively through inquiry-based instruction, informed by constructivist theory ( Ausubel, 1963 , 2000 ; Duch et al. , 2001 ; Nelson, 2008 ). In a survey of 103 instructors who taught college courses that included creativity instruction, Bull et al. (1995) asked respondents to rate the importance of various course characteristics for enhancing student creativity. Items ranking high on the list were: providing a social climate in which students feels safe, an open classroom environment that promotes tolerance for ambiguity and independence, the use of humor, metaphorical thinking, and problem defining. Many of the responses emphasized the same strategies as those advanced to promote creative problem solving (e.g., Mumford et al. , 1991 ; McFadzean, 2002 ; Treffinger and Isaksen, 2005 ) and critical thinking ( Abrami et al. , 2008 ).

In a careful meta-analysis, Scott et al. (2004) examined 70 instructional interventions designed to enhance and measure creative performance. The results were striking. Courses that stressed techniques such as critical thinking, convergent thinking, and constraint identification produced the largest positive effect sizes. More open techniques that provided less guidance in strategic approaches had less impact on the instructional outcomes. A striking finding was the effectiveness of being explicit; approaches that clearly informed students about the nature of creativity and offered clear strategies for creative thinking were most effective. Approaches such as social modeling, cooperative learning, and case-based (project-based) techniques that required the application of newly acquired knowledge were found to be positively correlated to high effect sizes. The most clear-cut result to emerge from the Scott et al. (2004) study was simply to confirm that creativity instruction can be highly successful in enhancing divergent thinking, problem solving, and imaginative performance. Most importantly, of the various cognitive processes examined, those linked to the generation of new ideas such as problem finding, conceptual combination, and idea generation showed the greatest improvement. The success of creativity instruction, the authors concluded, can be attributed to “developing and providing guidance concerning the application of requisite cognitive capacities … [and] a set of heuristics or strategies for working with already available knowledge” (p. 382).

Many of the scientific teaching practices that have been shown by research to foster content mastery and HOCS, and that are coming more widely into use, also would be consistent with promoting creativity. Wood (2009) has recently reviewed examples of such practices and how to apply them. These include relatively small modifications of the traditional lecture to engender more active learning, such as the use of concept tests and peer instruction ( Mazur, 1996 ), Just-in-Time-Teaching techniques ( Novak et al. , 1999 ), and student response systems known as “clickers” ( Knight and Wood, 2005 ; Crossgrove and Curran, 2008 ), all designed to allow the instructor to frequently and effortlessly elicit and respond to student thinking. Other strategies can transform the lecture hall into a workshop or studio classroom ( Gaffney et al. , 2008 ) where the teaching curriculum may emphasize problem-based (also known as project-based or case-based) learning strategies ( Duch et al. , 2001 ; Ebert-May and Hodder, 2008 ) or “community-based inquiry” in which students engage in research that enhances their critical-thinking skills ( Quitadamo et al. , 2008 ).

Another important approach that could readily subserve explicit creativity instruction is the use of computer-based interactive simulations, or “sims” ( Perkins and Wieman, 2008 ) to facilitate inquiry learning and effective, easy self-assessment. An example in the biological sciences would be Neurons in Action ( http://neuronsinaction.com/home/main ). In such educational environments, students gain conceptual understanding of scientific ideas through interactive engagement with materials (real or virtual), with each other, and with instructors. Following the tenets of scientific teaching, students are encouraged to pose and answer their own questions, to make sense of the materials, and to construct their own understanding. The question I pose here is whether an additional focus—guiding students to meet these challenges in a context that explicitly promotes creativity—would enhance learning and advance students' progress toward adaptive expertise?

Assessment of Creativity

To teach creativity, there must be measurable indicators to judge how much students have gained from instruction. Educational programs intended to teach creativity became popular after the Torrance Tests of Creative Thinking (TTCT) was introduced in the 1960s ( Torrance, 1974 ). But it soon became apparent that there were major problems in devising tests for creativity, both because of the difficulty of defining the construct and because of the number and complexity of elements that underlie it. Tests of intelligence and other personality characteristics on creative individuals revealed a host of related traits such as verbal fluency, metaphorical thinking, flexible decision making, tolerance of ambiguity, willingness to take risks, autonomy, divergent thinking, self-confidence, problem finding, ideational fluency, and belief in oneself as being “creative” ( Barron and Harrington, 1981 ; Tardif and Sternberg, 1988 ; Runco and Nemiro, 1994 ; Snyder et al. , 2004 ). Many of these traits have been the focus of extensive research of recent decades, but, as noted above, creativity is not defined by any one trait; there is now reason to believe that it is the interplay among the cognitive and affective processes that underlie inventiveness and the ability to find novel solutions to a problem.

Although the early creativity researchers recognized that assessing divergent thinking as a measure of creativity required tests for other underlying capacities ( Guilford, 1950 ; Torrance, 1974 ), these workers and their colleagues nonetheless believed that a high score for divergent thinking alone would correlate with real creative output. Unfortunately, no such correlation was shown ( Barron and Harrington, 1981 ). Results produced by many of the instruments initially designed to measure various aspects of creative thinking proved to be highly dependent on the test itself. A review of several hundred early studies showed that an individual's creativity score could be affected by simple test variables, for example, how the verbal pretest instructions were worded ( Barron and Harrington, 1981 , pp. 442–443). Most scholars now agree that divergent thinking, as originally defined, was not an adequate measure of creativity. The process of creative thinking requires a complex combination of elements that include cognitive flexibility, memory control, inhibitory control, and analogical thinking, enabling the mind to free-range and analogize, as well as to focus and test.

More recently, numerous psychometric measures have been developed and empirically tested (see Plucker and Renzulli, 1999 ) that allow more reliable and valid assessment of specific aspects of creativity. For example, the creativity quotient devised by Snyder et al. (2004) tests the ability of individuals to link different ideas and different categories of ideas into a novel synthesis. The Wallach–Kogan creativity test ( Wallach and Kogan, 1965 ) explores the uniqueness of ideas associated with a stimulus. For a more complete list and discussion, see the Creativity Tests website ( www.indiana.edu/∼bobweb/Handout/cretv_6.html ).

The most widely used measure of creativity is the TTCT, which has been modified four times since its original version in 1966 to take into account subsequent research. The TTCT-Verbal and the TTCT-Figural are two versions ( Torrance, 1998 ; see http://ststesting.com/2005giftttct.html ). The TTCT-Verbal consists of five tasks; the “stimulus” for each task is a picture to which the test-taker responds briefly in writing. A sample task that can be viewed from the TTCT Demonstrator website asks, “Suppose that people could transport themselves from place to place with just a wink of the eye or a twitch of the nose. What might be some things that would happen as a result? You have 3 min.” ( www.indiana.edu/∼bobweb/Handout/d3.ttct.htm ).

In the TTCT-Figural, participants are asked to construct a picture from a stimulus in the form of a partial line drawing given on the test sheet (see example below; Figure 1 ). Specific instructions are to “Add lines to the incomplete figures below to make pictures out of them. Try to tell complete stories with your pictures. Give your pictures titles. You have 3 min.” In the introductory materials, test-takers are urged to “… think of a picture or object that no one else will think of. Try to make it tell as complete and as interesting a story as you can …” ( Torrance et al. , 2008 , p. 2).

Figure 1. Sample figural test item from the TTCT Demonstrator website ( www.indiana.edu/∼bobweb/Handout/d3.ttct.htm ).

How would an instructor in a biology course judge the creativity of students' responses to such an item? To assist in this task, the TTCT has scoring and norming guides ( Torrance, 1998 ; Torrance et al. , 2008 ) with numerous samples and responses representing different levels of creativity. The guides show sample evaluations based upon specific indicators such as fluency, originality, elaboration (or complexity), unusual visualization, extending or breaking boundaries, humor, and imagery. These examples are easy to use and provide a high degree of validity and generalizability to the tests. The TTCT has been more intensively researched and analyzed than any other creativity instrument, and the norming samples have longitudinal validations and high predictive validity over a wide age range. In addition to global creativity scores, the TTCT is designed to provide outcome measures in various domains and thematic areas to allow for more insightful analysis ( Kaufman and Baer, 2006 ). Kim (2006) has examined the characteristics of the TTCT, including norms, reliability, and validity, and concludes that the test is an accurate measure of creativity. When properly used, it has been shown to be fair in terms of gender, race, community status, and language background. According to Kim (2006) and other authorities in the field ( McIntyre et al. , 2003 ; Scott et al. , 2004 ), Torrance's research and the development of the TTCT have provided groundwork for the idea that creative levels can be measured and then increased through instruction and practice.

SCIENTIFIC TEACHING TO PROMOTE CREATIVITY

How could creativity instruction be integrated into scientific teaching.

Guidelines for designing specific course units that emphasize HOCS by using strategies of scientific teaching are now available from the current literature. As an example, Karen Cloud-Hansen and colleagues ( Cloud-Hansen et al. , 2008 ) describe a course titled, “Ciprofloxacin Resistance in Neisseria gonorrhoeae .” They developed this undergraduate seminar to introduce college freshmen to important concepts in biology within a real-world context and to increase their content knowledge and critical-thinking skills. The centerpiece of the unit is a case study in which teams of students are challenged to take the role of a director of a local public health clinic. One of the county commissioners overseeing the clinic is an epidemiologist who wants to know “how you plan to address the emergence of ciprofloxacin resistance in Neisseria gonorrhoeae ” (p. 304). State budget cuts limit availability of expensive antibiotics and some laboratory tests to patients. Student teams are challenged to 1) develop a plan to address the medical, economic, and political questions such a clinic director would face in dealing with ciprofloxacin-resistant N. gonorrhoeae ; 2) provide scientific data to support their conclusions; and 3) describe their clinic plan in a one- to two-page referenced written report.

Throughout the 3-wk unit, in accordance with the principles of problem-based instruction ( Duch et al. , 2001 ), course instructors encourage students to seek, interpret, and synthesize their own information to the extent possible. Students have access to a variety of instructional formats, and active-learning experiences are incorporated throughout the unit. These activities are interspersed among minilectures and give the students opportunities to apply new information to their existing base of knowledge. The active-learning activities emphasize the key concepts of the minilectures and directly confront common misconceptions about antibiotic resistance, gene expression, and evolution. Weekly classes include question/answer/discussion sessions to address student misconceptions and 20-min minilectures on such topics as antibiotic resistance, evolution, and the central dogma of molecular biology. Students gather information about antibiotic resistance in N. gonorrhoeae , epidemiology of gonorrhea, and treatment options for the disease, and each team is expected to formulate a plan to address ciprofloxacin resistance in N. gonorrhoeae .

In this project, the authors assessed student gains in terms of content knowledge regarding topics covered such as the role of evolution in antibiotic resistance, mechanisms of gene expression, and the role of oncogenes in human disease. They also measured HOCS as gains in problem solving, according to a rubric that assessed self-reported abilities to communicate ideas logically, solve difficult problems about microbiology, propose hypotheses, analyze data, and draw conclusions. Comparing the pre- and posttests, students reported significant learning of scientific content. Among the thinking skill categories, students demonstrated measurable gains in their ability to solve problems about microbiology but the unit seemed to have little impact on their more general perceived problem-solving skills ( Cloud-Hansen et al. , 2008 ).

What would such a class look like with the addition of explicit creativity-promoting approaches? Would the gains in problem-solving abilities have been greater if during the minilectures and other activities, students had been introduced explicitly to elements of creative thinking from the Sternberg and Williams (1998) list described above? Would the students have reported greater gains if their instructors had encouraged idea generation with weekly brainstorming sessions; if they had reminded students to cross-fertilize ideas by integrating material across subject areas; built self-efficacy by helping students believe in their own capacity to be creative; helped students question their own assumptions; and encouraged students to imagine other viewpoints and possibilities? Of most relevance, could the authors have been more explicit in assessing the originality of the student plans? In an experiment that required college students to develop plans of a different, but comparable, type, Osborn and Mumford (2006) created an originality rubric ( Figure 2 ) that could apply equally to assist instructors in judging student plans in any course. With such modifications, would student gains in problem-solving abilities or other HOCS have been greater? Would their plans have been measurably more imaginative?

Figure 2. Originality rubric (adapted from Osburn and Mumford, 2006 , p. 183).

Answers to these questions can only be obtained when a course like that described by Cloud-Hansen et al. (2008) is taught with explicit instruction in creativity of the type I described above. But, such answers could be based upon more than subjective impressions of the course instructors. For example, students could be pretested with items from the TTCT-Verbal or TTCT-Figural like those shown. If, during minilectures and at every contact with instructors, students were repeatedly reminded and shown how to be as creative as possible, to integrate material across subject areas, to question their own assumptions and imagine other viewpoints and possibilities, would their scores on TTCT posttest items improve? Would the plans they formulated to address ciprofloxacin resistance become more imaginative?

Recall that in their meta-analysis, Scott et al. (2004) found that explicitly informing students about the nature of creativity and offering strategies for creative thinking were the most effective components of instruction. From their careful examination of 70 experimental studies, they concluded that approaches such as social modeling, cooperative learning, and case-based (project-based) techniques that required the application of newly acquired knowledge were positively correlated with high effect sizes. The study was clear in confirming that explicit creativity instruction can be successful in enhancing divergent thinking and problem solving. Would the same strategies work for courses in ecology and environmental biology, as detailed by Ebert-May and Hodder (2008) , or for a unit elaborated by Knight and Wood (2005) that applies classroom response clickers?

Finally, I return to my opening question with the fictional Dr. Dunne. Could a weekly brainstorming “invention session” included in a course like those described here serve as the site where students are introduced to concepts and strategies of creative problem solving? As frequently applied in schools of engineering ( Paulus and Nijstad, 2003 ), brainstorming provides an opportunity for the instructor to pose a problem and to ask the students to suggest as many solutions as possible in a brief period, thus enhancing ideational fluency. Here, students can be encouraged explicitly to build on the ideas of others and to think flexibly. Would brainstorming enhance students' divergent thinking or creative abilities as measured by TTCT items or an originality rubric? Many studies have demonstrated that group interactions such as brainstorming, under the right conditions, can indeed enhance creativity ( Paulus and Nijstad, 2003 ; Scott et al. , 2004 ), but there is little information from an undergraduate science classroom setting. Intellectual Ventures, a firm founded by Nathan Myhrvold, the creator of Microsoft's Research Division, has gathered groups of engineers and scientists around a table for day-long sessions to brainstorm about a prearranged topic. Here, the method seems to work. Since it was founded in 2000, Intellectual Ventures has filed hundreds of patent applications in more than 30 technology areas, applying the “invention session” strategy ( Gladwell, 2008 ). Currently, the company ranks among the top 50 worldwide in number of patent applications filed annually. Whether such a technique could be applied successfully in a college science course will only be revealed by future research.

• Abrami P. C., Bernard R. M., Borokhovski E., Wadem A., Surkes M. A., Tamim R., Zhang D. ( 2008 ). Instructional interventions affecting critical thinking skills and dispositions: a stage 1 meta-analysis . Rev. Educ. Res 78 , 1102-1134. Google Scholar
• Amabile T. M. ( 1996 ). Creativity in Context , Boulder, CO: Westview Press. Google Scholar
• Amabile T. M., Barsade S. G., Mueller J. S., Staw B. M. ( 2005 ). Affect and creativity at work . Admin. Sci. Q 50 , 367-403. Google Scholar
• Ausubel D. ( 1963 ). The Psychology of Meaningful Verbal Learning , New York: Grune and Stratton. Google Scholar
• Ausubel B. ( 2000 ). The Acquisition and Retention of Knowledge: A Cognitive View , Boston, MA: Kluwer Academic Publishers. Google Scholar
• Banaji S., Burn A., Buckingham D. ( 2006 ). The Rhetorics of Creativity: A Review of the Literature , accessed 29 December 2008 London: Centre for the Study of Children, Youth and Media, www.creativepartnerships.com/data/files/rhetorics-of-creativity-12.pdf . Google Scholar
• Barron F., Harrington D. M. ( 1981 ). Creativity, intelligence and personality . Ann. Rev. Psychol 32 , 439-476. Google Scholar
• Beller M. ( 1999 ). Quantum Dialogue: The Making of a Revolution , Chicago, IL: University of Chicago Press. Google Scholar
• Blair C., Razza R. P. ( 2007 ). Relating effortful control, executive function, and false belief understanding to emerging math and literacy ability in kindergarten . Child Dev 78 , 647-663. Medline ,  Google Scholar
• Bodrova E., Leong D. J. ( 2001 ). The Tool of the Mind: a case study of implementing the Vygotskian approach In: American Early Childhood and Primary Classrooms , Geneva, Switzerland: UNESCO International Bureau of Education. Google Scholar
• Bransford J. D.Brown A. L.Cocking R. R. ( 2000 ). How People Learn: Brain, Mind, Experience, and School , Washington, DC: National Academies Press. Google Scholar
• Brophy D. R. ( 2006 ). A comparison of individual and group efforts to creatively solve contrasting types of problems . Creativity Res. J 18 , 293-315. Google Scholar
• Bruner J. ( 1965 ). The growth of mind . Am. Psychol 20 , 1007-1017. Medline ,  Google Scholar
• Bull K. S., Montgomery D., Baloche L. ( 1995 ). Teaching creativity at the college level: a synthesis of curricular components perceived as important by instructors . Creativity Res. J 8 , 83-90. Google Scholar
• Burton R. ( 2008 ). On Being Certain: Believing You Are Right Even When You're Not , New York: St. Martin's Press. Google Scholar
• Cloud-Hanson K. A., Kuehner J. N., Tong L., Miller S., Handelsman J. ( 2008 ). Money, sex and drugs: a case study to teach the genetics of antibiotic resistance . CBE Life Sci. Educ 7 , 302-309. Medline ,  Google Scholar
• Craft A. ( 2000 ). Teaching Creativity: Philosophy and Practice , New York: Routledge. Google Scholar
• Crawford V. M. ( 2007 ). Adaptive expertise as knowledge building in science teachers' problem solving accessed 1 July 2008 Proceedings of the Second European Cognitive Science Conference Delphi, Greece http://ctl.sri.com/publications/downloads/Crawford_EuroCogSci07Proceedings.pdf . Google Scholar
• Crawford V. M., Brophy S. ( 2006 ). Adaptive Expertise: Theory, Methods, Findings, and Emerging Issues; September 2006 In: accessed 1 July 2008 Menlo Park, CA: SRI International, http://ctl.sri.com/publications/downloads/AESymposiumReportOct06.pdf . Google Scholar
• Crossgrove K., Curran K. L. ( 2008 ). Using clickers in nonmajors- and majors-level biology courses: student opinion, learning, and long-term retention of course material . CBE Life Sci. Educ 7 , 146-154. Link ,  Google Scholar
• Crowe A., Dirks C., Wenderoth M. P. ( 2008 ). Biology in bloom: implementing Bloom's taxonomy to enhance student learning in biology . CBE Life Sci. Educ 7 , 368-381. Link ,  Google Scholar
• Davidson M. C., Amso D., Anderson L. C., Diamond A. ( 2006 ). Development of cognitive control and executive functions from 4–13 years: evidence from manipulations of memory, inhibition, and task switching . Neuropsychologia 44 , 2037-2078. Medline ,  Google Scholar
• DeHaan R. L. ( 2005 ). The impending revolution in undergraduate science education . J. Sci. Educ. Technol 14 , 253-270. Google Scholar
• Diamond A., Barnett W. S., Thomas J., Munro S. ( 2007 ). Preschool program improves cognitive control . Science 318 , 1387-1388. Medline ,  Google Scholar
• Duch B. J., Groh S. E., Allen D. E. ( 2001 ). The Power of Problem-based Learning , Sterling, VA: Stylus Publishers. Google Scholar
• Durston S., Davidson M. C., Thomas K. M., Worden M. S., Tottenham N., Martinez A., Watts R., Ulug A. M., Caseya B. J. ( 2003 ). Parametric manipulation of conflict and response competition using rapid mixed-trial event-related fMRI . Neuroimage 20 , 2135-2141. Medline ,  Google Scholar
• Ebert-May D., Hodder J. ( 2008 ). Pathways to Scientific Teaching , Sunderland, MA: Sinauer. Google Scholar
• Finke R. A., Ward T. B., Smith S. M. ( 1996 ). Creative Cognition: Theory, Research and Applications , Boston, MA: MIT Press. Google Scholar
• Freeman S., O'Connor E., Parks J. W., Cunningham M., Hurley D., Haak D., Dirks C., Wenderoth M. P. ( 2007 ). Prescribed active learning increases performance in introductory biology . CBE Life Sci. Educ 6 , 132-139. Link ,  Google Scholar
• Gabora L. ( 2002 ). Hewett T.Kavanagh E. Cognitive mechanisms underlying the creative process Proceedings of the Fourth International Conference on Creativity and Cognition 2002 October 13–16 Loughborough University, United Kingdom 126-133. Google Scholar
• Gaffney J.D.H., Richards E., Kustusch M. B., Ding L., Beichner R. ( 2008 ). Scaling up education reform . J. Coll. Sci. Teach 37 , 48-53. Google Scholar
• Gardner H. ( 1993 ). Creating Minds: An Anatomy of Creativity Seen through the Lives of Freud, Einstein, Picasso, Stravinsky, Eliot, Graham, and Ghandi In: New York: Harper Collins. Google Scholar
• Gladwell M. ( 2008 ). In the air; who says big ideas are rare? The New Yorker accessed 19 May 2008 www.newyorker.com/reporting/2008/05/12/080512fa_fact_gladwell . Google Scholar
• Guilford J. P. ( 1950 ). Creativity . Am. Psychol 5 , 444-454. Medline ,  Google Scholar
• Hake R. ( 2005 ). The physics education reform effort: a possible model for higher education . Natl. Teach. Learn. Forum 15 , 1-6. Google Scholar
• Halpern D. E., Hakel M. D. ( 2003 ). Applying the science of learning to the university and beyond . Change 35 , 36-42. Google Scholar
• Handelsman J. ( 2004 ). Scientific teaching . Science 304 , 521-522. Medline ,  Google Scholar
• Handelsman J, Miller S., Pfund C. ( 2007 ). Scientific Teaching , New York: W. H. Freeman and Co. Google Scholar
• Haring-Smith T. ( 2006 ). Creativity research review: some lessons for higher education. Association of American Colleges and Universities . Peer Rev 8 , 23-27. Google Scholar
• Hatano G., Ouro Y. ( 2003 ). Commentary: reconceptualizing school learning using insight from expertise research . Educ. Res 32 , 26-29. Google Scholar
• Hrepic Z., Zollman D. A., Rebello N. S. ( 2007 ). Comparing students' and experts' understanding of the content of a lecture . J. Sci. Educ. Technol 16 , 213-224. Google Scholar
• Hunsaker S. L. ( 2005 ). Outcomes of creativity training programs . Gifted Child Q 49 , 292-298. Google Scholar
• Kaufman J. C., Baer J. ( 2006 ). Intelligent testing with Torrance . Creativity Res. J 18 , 99-102. Google Scholar
• Kaufman J. C., Beghetto R. A. ( 2008 , Ed. R. L. DeHaanK.M.V. Narayan , Exploring mini-C: creativity across cultures In: Education for Innovation: Implications for India, China and America , Rotterdam, The Netherlands: Sense Publishers, 165-180. Google Scholar
• Kaufman J. C., Sternberg R. J. ( 2007 ). Creativity . Change 39 , 55-58. Google Scholar
• Kim K. H. ( 2006 ). Can we trust creativity tests: a review of the Torrance Tests of Creative Thinking (TTCT) . Creativity Res. J 18 , 3-14. Google Scholar
• Knight J. K., Wood W. B. ( 2005 ). Teaching more by lecturing less . Cell Biol. Educ 4 , 298-310. Link ,  Google Scholar
• Cetina Knorr K. ( 1995 , Ed. S. JasanoffG. MarkleJ. PetersenT. Pinch , Laboratory studies: the cultural approach to the study of science In: Handbook of Science and Technology Studies , Thousand Oaks, CA: Sage Publications, 140-166. Google Scholar
• Koestler A. ( 1964 ). The Act of Creation , New York: Macmillan. Google Scholar
• Latour B., Woolgar S. ( 1986 ). Laboratory Life: The Construction of Scientific Facts , Princeton, NJ: Princeton University Press. Google Scholar
• MacKinnon D. W. ( 1978 , Ed. D. W. MacKinnon , What makes a person creative? In: In Search of Human Effectiveness , New York: Universe Books, 178-186. Google Scholar
• Martindale C. ( 1999 , Ed. R. J. Sternberg , Biological basis of creativity In: Handbook of Creativity , Cambridge, United Kingdom: Cambridge University Press, 137-152. Google Scholar
• Mazur E. ( 1996 ). Peer Instruction: A User's Manual , Upper Saddle River, NJ: Prentice Hall. Google Scholar
• McFadzean E. ( 2002 ). Developing and supporting creative problem-solving teams: Part 1—a conceptual model . Manage. Decis 40 , 463-475. Google Scholar
• McGregor G. D. ( 2001 ). Creative thinking instruction for a college study skills program: a case study. . Dissert Abstr. Intl 62 , 3293A UMI No. AAT 3027933. Google Scholar
• McIntyre F. S., Hite R. E., Rickard M. K. ( 2003 ). Individual characteristics and creativity in the marketing classroom: exploratory insights . J. Mark. Educ 25 , 143-149. Google Scholar
• Mestre J. P. ( 2005 ). Transfer of Learning: From a Modern Multidisciplinary Perspective , Greenwich, CT: Information Age Publishing. Google Scholar
• Mumford M. D., Mobley M. I., Uhlman C. E., Reiter-Palmon R., Doares L. M. ( 1991 ). Process analytic models of creative capacities . Creativity Res. J 4 , 91-122. Google Scholar
• National Research Council ( 2007 ). Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future, Committee on Science, Engineering and Public Policy In: Washington, DC: National Academies Press. Google Scholar
• Neisser U. ( 1963 ). The multiplicity of thought . Br. J. Psychol 54 , 1-14. Medline ,  Google Scholar
• Nelson C. E. ( 2008 ). Teaching evolution (and all of biology) more effectively: strategies for engagement, critical reasoning, and confronting misconceptions Integrative and Comparative Biology Advance Access accessed 15 September 2008 http://icb.oxfordjournals.org/cgi/reprint/icn027v1.pdf . Google Scholar
• Novak G, Gavrin A., Christian W, Patterson E. ( 1999 ). Just-in-Time Teaching: Blending Active Learning with Web Technology , San Francisco, CA: Pearson Benjamin Cummings. Google Scholar
• Osborn A. F. ( 1948 ). Your Creative Power , New York: Scribner. Google Scholar
• Osborn A. F. ( 1979 ). Applied Imagination , New York: Scribner. Google Scholar
• Osburn H. K., Mumford M. D. ( 2006 ). Creativity and planning: training interventions to develop creative problem-solving skills . Creativity Res. J 18 , 173-190. Google Scholar
• Paulus P. B., Nijstad B. A. ( 2003 ). Group Creativity: Innovation through Collaboration , New York: Oxford University Press. Google Scholar
• Perkins K. K., Wieman C. E. ( 2008 , Ed. R. L. DeHaanK.M.V. Narayan , Innovative teaching to promote innovative thinking In: Education for Innovation: Implications for India, China and America , Rotterdam, The Netherlands: Sense Publishers, 181-210. Google Scholar
• Plucker J. A., Renzulli J. S. ( 1999 , Ed. R. J. Sternberg , Psychometric approaches to the study of human creativity In: Handbook of Creativity , Cambridge, United Kingdom: Cambridge University Press, 35-61. Google Scholar
• Quitadamo I. J., Faiola C. L., Johnson J. E., Kurtz M. J. ( 2008 ). Community-based inquiry improves critical thinking in general education biology . CBE Life Sci. Educ 7 , 327-337. Link ,  Google Scholar
• Runco M. A. ( 2004 ). Creativity . Annu. Rev. Psychol 55 , 657-687. Medline ,  Google Scholar
• Runco M. A., Nemiro J. ( 1994 ). Problem finding, creativity, and giftedness . Roeper Rev 16 , 235-241. Google Scholar
• Sawyer R. K. ( 2005 ). Educating for Innovation Thinking Skills Creativity accessed 13 August 2008 1 41-48 www.artsci.wustl.edu/∼ksawyer/PDFs/Thinkjournal.pdf . Google Scholar
• Sawyer R. K. ( 2006 ). Explaining Creativity: The Science of Human Innovation , New York: Oxford University Press. Google Scholar
• Schwartz D. L., Bransford J. D., Sears D. ( 2005 , Ed. J. P. Mestre , Efficiency and innovation in transfer In: Transfer of Learning from a Modern Multidisciplinary Perspective , Greenwich, CT: Information Age Publishing, 1-51. Google Scholar
• Scott G., Leritz L. E., Mumford M. D. ( 2004 ). The effectiveness of creativity training: a quantitative review . Creativity Res. J 16 , 361-388. Google Scholar
• Simonton D. K. ( 1975 ). Sociocultural context of individual creativity: a transhistorical time-series analysis . J. Pers. Soc. Psychol 32 , 1119-1133. Medline ,  Google Scholar
• Simonton D. K. ( 2004 ). Creativity in Science: Chance, Logic, Genius, and Zeitgeist , Oxford, United Kingdom: Cambridge University Press. Google Scholar
• Sloman S. ( 1996 ). The empirical case for two systems of reasoning . Psychol. Bull 9 , 3-22. Google Scholar
• Smith G. F. ( 1998 ). Idea generation techniques: a formulary of active ingredients . J. Creative Behav 32 , 107-134. Google Scholar
• Snyder A., Mitchell J., Bossomaier T., Pallier G. ( 2004 ). The creativity quotient: an objective scoring of ideational fluency . Creativity Res. J 16 , 415-420. Google Scholar
• Sternberg R. J. ( 2003 ). What is an “expert student?” . Educ. Res. 32 , 5-9. Google Scholar
• Sternberg R., Williams W. M. ( 1998 ). Teaching for creativity: two dozen tips accessed 25 March 2008 www.cdl.org/resource-library/articles/teaching_creativity.php . Google Scholar
• Tardif T. Z., Sternberg R. J. ( 1988 , Ed. R. J. Sternberg , What do we know about creativity? In: The Nature of Creativity , New York: Cambridge University Press, 429-440. Google Scholar
• Torrance E. P. ( 1974 ). Norms and Technical Manual for the Torrance Tests of Creative Thinking , Bensenville, IL: Scholastic Testing Service. Google Scholar
• Torrance E. P. ( 1998 ). The Torrance Tests of Creative Thinking Norms—Technical Manual Figural (Streamlined) Forms A and B , Bensenville, IL: Scholastic Testing Service. Google Scholar
• Torrance E. P., Ball O. E., Safter H. T. ( 2008 ). Torrance Tests of Creative Thinking: Streamlined Scoring Guide for Figural Forms A and B , Bensenville, IL: Scholastic Testing Service. Google Scholar
• Treffinger D. J., Isaksen S. G. ( 2005 ). Creative problem solving: the history, development, and implications for gifted education and talent development . Gifted Child Q 49 , 342-357. Google Scholar
• Vandervert L. R., Schimpf P. H., Liu H. ( 2007 ). How working memory and the cerebellum collaborate to produce creativity and innovation . Creativity Res. J 9 , 1-18. Google Scholar
• Wallach M. A., Kogan N. ( 1965 ). Modes of Thinking in Young Children: A Study of the Creativity-Intelligence Distinction , New York: Holt, Rinehart and Winston. Google Scholar
• Wood W. B. ( 2009 ). Innovations in undergraduate biology teaching and why we need them . Annu. Rev. Cell Dev. Biol in press. Medline ,  Google Scholar
• Research-based learning as an innovative approach for teaching students of environmental engineering: a case study of the emerging field of microplastics in soil 15 July 2024 | Discover Education, Vol. 3, No. 1
• ChatGPT improves creative problem-solving performance in university students: An experimental study 1 Jul 2024 | Computers & Education, Vol. 215
• Are we teaching novice instructional designers to be creative? A qualitative case study 16 January 2024 | Instructional Science, Vol. 52, No. 3
• Educators as agents of breadth-biased learning: using social reconstructionism as rationale for embracing media multitasking and enhancing teaching practices in higher education 3 April 2024 | Frontiers in Psychology, Vol. 15
• Searching for creativity: How people search to generate new ideas 1 December 2023 | Journal of the Association for Information Science and Technology, Vol. 75, No. 4
• Fostering creativity in low-engagement students through socratic dialogue: An experiment in an operations class 1 Mar 2024 | The International Journal of Management Education, Vol. 22, No. 1
• Cognitive flexibility and academic performance: Individual and cross-national patterns among adolescents in 57 countries 1 Feb 2024 | Personality and Individual Differences, Vol. 217
• What's in a word? Student beliefs and understanding about green chemistry 1 January 2024 | Chemistry Education Research and Practice, Vol. 25, No. 1
• Preservice Physical Education Teachers’ Resistance to Change: The Importance of Occupational Socialization Experiences 26 October 2023 | Trends in Higher Education, Vol. 2, No. 4
• How can we measure metacognition in creative problem-solving? Standardization of the MCPS scale 1 Sep 2023 | Thinking Skills and Creativity, Vol. 49
• TOWARDS ENHANCING CREATIVITY AND INNOVATION IN EDUCATION SYSTEM FOR YOUTH IN HAIL REGION 1 August 2023 | Advanced Education, Vol. 10, No. 22
• Investigating the impact of innovation competence instruction in higher engineering education 12 June 2023 | European Journal of Engineering Education, Vol. 10
• Regaining creativity in science: insights from conversation 17 May 2023 | Royal Society Open Science, Vol. 10, No. 5
• How transdisciplinary integration, creativity and student motivation interact in three STEAM projects for gifted education? 29 March 2023 | Gifted Education International, Vol. 39, No. 2
• Does creative coursework predict educational, career, and community engagement outcomes for arts alumni? 15 November 2022 | Creativity Research Journal, Vol. 35, No. 2
• Make science disruptive again 27 March 2023 | Nature Biotechnology, Vol. 41, No. 4
• Promoting Creativity in Undergraduate Recreation and Leisure Services Classrooms: An Overview 19 March 2021 | SCHOLE: A Journal of Leisure Studies and Recreation Education, Vol. 38, No. 1
• A Positive Association between Working Memory Capacity and Human Creativity: A Meta-Analytic Evidence 13 January 2023 | Journal of Intelligence, Vol. 11, No. 1
• Students Creativity Through Digital Mind Map 26 July 2023
• Pedagogical and School Practices to Foster Key Competences and Domain-General Literacy 23 August 2023
• Coping with Challenges and Uncertainty in Scientific Research 13 Dec 2022 | Asia-Pacific Science Education, Vol. 8, No. 2
• Teaching design thinking as a tool to address complex public health challenges in public health students: a case study 12 April 2022 | BMC Medical Education, Vol. 22, No. 1
• ‘Allowing them to dream’: fostering creativity in mathematics undergraduates 26 May 2022 | Journal of Further and Higher Education, Vol. 46, No. 10
• Interaction with metaphors enhances creative potential 1 July 2022 | Journal of Poetry Therapy, Vol. 35, No. 4
• Creative problem solving in knowledge-rich contexts 1 Oct 2022 | Trends in Cognitive Sciences, Vol. 26, No. 10
• Teaching Protein–Ligand Interactions Using a Case Study on Tau in Alzheimer’s Disease 1 July 2022 | Journal of Chemical Education, Vol. 99, No. 8
• The Effectiveness of Collaborative Learning on Critical Thinking, Creative Thinking, and Metacognitive Skill Ability: Meta-Analysis on Biological Learning 15 July 2022 | European Journal of Educational Research, Vol. volume-11-2022, No. volume-11-issue-3-july-2022
• Student approaches to creative processes when participating in an open-ended project in science 30 June 2022 | International Journal of Science Education, Vol. 44, No. 10
• Arts, Machines, and Creative Education 24 Jun 2022
• Perceived Learning Effectiveness and Student Satisfaction 6 May 2022
• A Contribution to Scientific Creativity: A Validation Study Measuring Divergent Problem Solving Ability 3 September 2021 | Creativity Research Journal, Vol. 34, No. 2
• Problem Solving and Digital Transformation: Acquiring Skills through Pretend Play in Kindergarten 28 January 2022 | Education Sciences, Vol. 12, No. 2
• Growing Innovation and Collaboration Through Assessment and Feedback: A Toolkit for Assessing and Developing Students’ Soft Skills in Biological Experimentation 12 May 2022
• The Role of Creativity in Teaching Mathematics Online 1 December 2022
• Breathing Life Into Marketing Scholarship Through Creativity Learning and Teaching 1 Jan 2022
• Mobilizing Research-Based Learning (RBL) in Higher Education 1 Jan 2022
• Trends and opportunities by fostering creativity in science and engineering: a systematic review 2 September 2021 | European Journal of Engineering Education, Vol. 46, No. 6
• The effect of a scientific board game on improving creative problem solving skills 1 Sep 2021 | Thinking Skills and Creativity, Vol. 41
• Create Teaching Creativity through Training Management, Effectiveness Training, and Teacher Quality in the Covid-19 Pandemic 6 August 2021 | Journal of Ethnic and Cultural Studies, Vol. 8, No. 4
• Creativity and technology in teaching and learning: a literature review of the uneasy space of implementation  11 January 2021 | Educational Technology Research and Development, Vol. 69, No. 4
• Üstün Yetenekli Öğrencilerin Bilimsel Yaratıcılık ve Bilimsel Problem Çözme ile İlgili Öz Değerlendirmeleri 15 July 2021 | Yuzunci Yil Universitesi Egitim Fakultesi Dergisi
• Cultivating creative thinking in engineering student teams: Can a computer‐mediated virtual laboratory help? 23 November 2020 | Journal of Computer Assisted Learning, Vol. 37, No. 2
• Entrepreneurial competencies of undergraduate students: The case of universities in Nigeria 1 Mar 2021 | The International Journal of Management Education, Vol. 19, No. 1
• Promoting Creativity in General Education Mathematics Courses 5 August 2019 | PRIMUS, Vol. 31, No. 1
• Methodological Considerations for Understanding Students’ Problem Solving Processes and Affective Trajectories During Game-Based Learning: A Data Fusion Approach 3 July 2021
• The main trends in the process of building the creative potential of engineering students 18 June 2021 | E3S Web of Conferences, Vol. 274
• Research on the Present Situation and Countermeasures of Cultivating Graduate Students’ Innovation Ability under Cooperative Innovation Environment 1 Jan 2021 | Creative Education Studies, Vol. 09, No. 02
• Innovation Centers and the Information Schools: The Influence of LIS Faculty 1 Dec 2020 | Journal of Education for Library and Information Science, Vol. 61, No. 4
• Biomimetics: teaching the tools of the trade 28 September 2020 | FEBS Open Bio, Vol. 10, No. 11
• STEM academic teachers’ experiences of undertaking authentic assessment-led reform: a mixed method approach 21 March 2019 | Studies in Higher Education, Vol. 45, No. 9
• Problems of forming marketing competencies in the digital economy 1 Sep 2020 | IOP Conference Series: Materials Science and Engineering, Vol. 940, No. 1
• Culturally Responsive Assessment of Physical Science Skills and Abilities: Development, Field Testing, Implementation, and Results 2 June 2020 | Journal of Advanced Academics, Vol. 31, No. 3
• Culturally Responsive Assessment of Life Science Skills and Abilities: Development, Field Testing, Implementation, and Results 3 June 2020 | Journal of Advanced Academics, Vol. 31, No. 3
• Curriculum Differentiation’s Capacity to Extend Gifted Students in Secondary Mixed-ability Science Classes 27 June 2020 | Talent, Vol. 10, No. 1
• Student Motivation from and Resistance to Active Learning Rooted in Essential Science Practices 23 December 2017 | Research in Science Education, Vol. 50, No. 1
• Integrating Entrepreneurship and Art to Improve Creative Problem Solving in Fisheries Education 27 February 2020 | Fisheries, Vol. 45, No. 2
• Concepts Re-imagined: Relational Signs Beyond Definitional Rigidity 15 August 2020
• Promoting Student Creativity and Inventiveness in Science and Engineering 24 October 2020
• Teaching for Leadership, Innovation, and Creativity 1 Jan 2020
• Problem Çözme Becerileri Eğitim Programının Çocukların Karar Verme Becerileri Üzerindeki Etkisi 30 December 2019 | Erzincan Üniversitesi Eğitim Fakültesi Dergisi, Vol. 21, No. 3
• Mento’s change model in teaching competency-based medical education 27 December 2019 | BMC Medical Education, Vol. 19, No. 1
• The Effects of Individual Preparations on Group Creativity 21 January 2020
• The Value of Creativity for Enhancing Translational Ecologies, Insights, and Discoveries 9 July 2019 | Frontiers in Psychology, Vol. 10
• Using the International Classification of Functioning, Disability, and Health to Guide Students' Clinical Approach to Aging With Pathology 1 Jul 2019 | Topics in Geriatric Rehabilitation, Vol. 35, No. 3
• Impact of Brainstorming Strategy in Dealing With Knowledge Retention Skill: An Insight Into Special Learners' Needs In Saudi Arabia 1 January 2021 | MIER Journal of Educational Studies Trends & Practices
• The potential of students’ creative disposition as a perspective to develop creative teaching and learning for senior high school biological science 12 March 2019 | Journal of Physics: Conference Series, Vol. 1157
• Evaluating Remote Experiment from a Divergent Thinking Point of View 25 July 2018
• Exploring Creative Education Practices and Implications: A Case study of National Chengchi University, Taiwan 4 April 2022 | Journal of Business and Economic Analysis, Vol. 02, No. 02
• Exploring Creative Education Practices and Implications: A Case study of National Chengchi University, Taiwan 1 January 2020 | Journal of Business and Economic Analysis, Vol. 02, No. 02
• Diverging from the Dogma: A Call to Train Creative Thinkers in Science 14 September 2018 | The Bulletin of the Ecological Society of America, Vol. 100, No. 1
• Comparison of German and Japanese student teachers’ views on creativity in chemistry class 16 May 2018 | Asia-Pacific Science Education, Vol. 4, No. 1
• The use of humour during a collaborative inquiry 27 August 2018 | International Journal of Science Education, Vol. 40, No. 14
• Connecting creative coursework exposure and college student engagement across academic disciplines 29 August 2019 | Gifted and Talented International, Vol. 33, No. 1-2
• Views of German chemistry teachers on creativity in chemistry classes and in general 1 January 2018 | Chemistry Education Research and Practice, Vol. 19, No. 3
• Embedding Critical and Creative Thinking in Chemical Engineering Practice 1 Jul 2018
• Introducing storytelling to educational robotic activities 1 Apr 2018
• Investigating Undergraduates’ Perceptions of Science in Courses Taught Using the CREATE Strategy 1 Mar 2018 | Journal of Microbiology & Biology Education, Vol. 19, No. 1
• Teachers’ learning on the workshop of STS approach as a way of enhancing inventive thinking skills 1 Jan 2018
• Creativity Development Through Inquiry-Based Learning in Biomedical Sciences 1 Jan 2018
• The right tool for the right task: Structured techniques prove less effective on an ill-defined problem finding task 1 Dec 2017 | Thinking Skills and Creativity, Vol. 26
• The influential factors and hierarchical structure of college students’ creative capabilities—An empirical study in Taiwan 1 Dec 2017 | Thinking Skills and Creativity, Vol. 26
• Teacher perceptions of professional role and innovative teaching at elementary schools in Taiwan 10 November 2017 | Educational Research and Reviews, Vol. 12, No. 21
• Evaluation of creative problem-solving abilities in undergraduate structural engineers through interdisciplinary problem-based learning 28 July 2016 | European Journal of Engineering Education, Vol. 42, No. 6
• What Shall I Write Next? 19 September 2017
• Inquiry-based Laboratory Activities on Drugs Analysis for High School Chemistry Learning 3 October 2017 | Journal of Physics: Conference Series, Vol. 895
• BARRIERS TO STUDENTS’ CREATIVE EVALUATION OF UNEXPECTED EXPERIMENTAL FINDINGS 25 June 2017 | Journal of Baltic Science Education, Vol. 16, No. 3
• Kyle J. Frantz ,
• Melissa K. Demetrikopoulos ,
• Shari L. Britner ,
• Laura L. Carruth ,
• Brian A. Williams ,
• John L. Pecore ,
• Robert L. DeHaan , and
• Christopher T. Goode
• Elizabeth Ambos, Monitoring Editor
• A present absence: undergraduate course outlines and the development of student creativity across disciplines 3 October 2016 | Teaching in Higher Education, Vol. 22, No. 2
• Exploring differences in creativity across academic majors for high-ability college students 16 February 2018 | Gifted and Talented International, Vol. 32, No. 1
• Creativity in chemistry class and in general – German student teachers’ views 1 January 2017 | Chemistry Education Research and Practice, Vol. 18, No. 2
• IMPORTANCE OF CREATIVITY IN ENTREPRENEURSHIP 1 January 2017
• Accessing the Finest Minds 1 Jan 2017
• Science and Innovative Thinking for Technical and Organizational Development 1 Jan 2017
• Learning High School Biology in a Social Context 1 Jan 2017 | Creative Education, Vol. 08, No. 15
• Possibilities and limitations of integrating peer instruction into technical creativity education 6 September 2016 | Instructional Science, Vol. 44, No. 6
• Creative Cognitive Processes in Higher Education 20 November 2014 | The Journal of Creative Behavior, Vol. 50, No. 4
• An Evidence-Based Review of Creative Problem Solving Tools 6 April 2016 | Human Resource Development Review, Vol. 15, No. 2
• Case-based exams for learning and assessment: Experiences in an information systems course 1 Apr 2016
• Case exams for assessing higher order learning: A comparative social media analytics usage exam 1 Apr 2016
• Beyond belief: Structured techniques prove more effective than a placebo intervention in a problem construction task 1 Mar 2016 | Thinking Skills and Creativity, Vol. 19
• A Belief System at the Core of Learning Science 1 Jan 2016
• Science and Innovative Thinking for Technical and Organizational Development 1 Jan 2016
• Student Research Work and Modeled Situations in Order to Bridge the Gap between Basic Science Concepts and Those from Preventive and Clinical Practice. Meaningful Learning and Informed beneficience 1 Jan 2016 | Creative Education, Vol. 07, No. 07
• FOSTERING FIFTH GRADERS’ SCIENTIFIC CREATIVITY THROUGH PROBLEM-BASED LEARNING 25 October 2015 | Journal of Baltic Science Education, Vol. 14, No. 5
• Scaffolding for Creative Product Possibilities in a Design-Based STEM Activity 16 November 2014 | Research in Science Education, Vol. 45, No. 5
• Intuition and insight: two concepts that illuminate the tacit in science education 18 June 2015 | Studies in Science Education, Vol. 51, No. 2
• Arts and crafts as adjuncts to STEM education to foster creativity in gifted and talented students 28 March 2015 | Asia Pacific Education Review, Vol. 16, No. 2
• Initiatives Towards an Education for Creativity 1 May 2015 | Procedia - Social and Behavioral Sciences, Vol. 180
• Brian A. Couch ,
• Tanya L. Brown ,
• Tyler J. Schelpat ,
• Mark J. Graham , and
• Jennifer K. Knight
• Michèle Shuster, Monitoring Editor
• Kim Quillin , and
• Stephen Thomas
• Mary Lee Ledbetter, Monitoring Editor
• The Design of IdeaWorks: Applying Social Learning Networks to Support Tertiary Education 21 July 2015
• References 1 Jan 2015
• Video Games and Malevolent Creativity 1 Jan 2015
• Modelling a Laboratory for Ideas as a New Tool for Fostering Engineering Creativity 1 Jan 2015 | Procedia Engineering, Vol. 100
• “Development of Thinking Skills” Course: Teaching TRIZ in Academic Setting 1 Jan 2015 | Procedia Engineering, Vol. 131
• Leadership in the Future Experts’ Creativity Development with Scientific Research Activities 4 November 2014
• Developing Deaf Children's Conceptual Understanding and Scientific Argumentation Skills: A Literature Review 3 January 2014 | Deafness & Education International, Vol. 16, No. 3
• Leslie M. Stevens , and
• Sally G. Hoskins
• Nancy Pelaez, Monitoring Editor
• 2014 | Cortex, Vol. 51
• A Sociotechnological Theory of Discursive Change and Entrepreneurial Capacity: Novelty and Networks 1 Jan 2014 | SSRN Electronic Journal, Vol. 3
• GEOverse: An Undergraduate Research Journal: Research Dissemination Within and Beyond the Curriculum 1 August 2013
• Learning by Practice, High-Pressure Student Ateliers 2 August 2013
• Relating Inter-Individual Differences in Verbal Creative Thinking to Cerebral Structures: An Optimal Voxel-Based Morphometry Study 5 November 2013 | PLoS ONE, Vol. 8, No. 11
• 21st Century Biology: An Interdisciplinary Approach of Biology, Technology, Engineering and Mathematics Education 1 Nov 2013 | Procedia - Social and Behavioral Sciences, Vol. 102
• Reclaiming creativity in the era of impact: exploring ideas about creative research in science and engineering 1 Nov 2013 | Studies in Higher Education, Vol. 38, No. 9
• An Evaluation of Alternative Ways of Computing the Creativity Quotient in a Design School Sample 1 Jul 2013 | Creativity Research Journal, Vol. 25, No. 3
• A.-M. Hoskinson ,
• M. D. Caballero , and
• J. K. Knight
• Eric Brewe, Monitoring Editor
• Understanding, attitude and environment 17 May 2013 | International Journal for Researcher Development, Vol. 4, No. 1
• Promoting Student Creativity and Inventiveness in Science and Engineering 1 Jan 2013
• Building creative thinking in the classroom: From research to practice 1 Jan 2013 | International Journal of Educational Research, Vol. 62
• A Demonstration of a Mastery Goal Driven Learning Environment to Foster Creativity in Engineering Design 1 Jan 2013 | SSRN Electronic Journal, Vol. 111
• The development of creative cognition across adolescence: distinct trajectories for insight and divergent thinking 8 October 2012 | Developmental Science, Vol. 16, No. 1
• A CROSS-NATIONAL STUDY OF PROSPECTIVE ELEMENTARY AND SCIENCE TEACHERS’ CREATIVITY STYLES 10 September 2012 | Journal of Baltic Science Education, Vol. 11, No. 3
• Evaluation of fostering students' creativity in preparing aided recalls for revision courses using electronic revision and recapitulation tools 2.0 1 Aug 2012 | Behaviour & Information Technology, Vol. 31, No. 8
• Scientific Creativity: The Missing Ingredient in Slovenian Science Education 15 April 2012 | European Journal of Educational Research, Vol. volume-1-2012, No. volume1-issue2.html
• Could the ‘evolution’ from biology to life sciences prevent ‘extinction’ of the subject field? 6 March 2012 | Suid-Afrikaanse Tydskrif vir Natuurwetenskap en Tegnologie, Vol. 31, No. 1
• Mobile innovations, executive functions, and educational developments in conflict zones: a case study from Palestine 1 October 2011 | Educational Technology Research and Development, Vol. 60, No. 1
• Informing Pedagogy Through the Brain-Targeted Teaching Model 1 Jan 2012 | Journal of Microbiology & Biology Education, Vol. 13, No. 1
• Teaching Creative Science Thinking 16 Dec 2011 | Science, Vol. 334, No. 6062
• Sally G. Hoskins ,
• David Lopatto , and
• Leslie M. Stevens
• Diane K. O'Dowd, Monitoring Editor
• Embedding Research-Based Learning Early in the Undergraduate Geography Curriculum 1 Aug 2011 | Journal of Geography in Higher Education, Vol. 35, No. 3
• Jared L. Taylor ,
• Karen M. Smith ,
• Adrian P. van Stolk , and
• George B. Spiegelman
• Debra Tomanek, Monitoring Editor
• 2010 | IFAC Proceedings Volumes, Vol. 43, No. 17
• Critical and Creative Thinking Activities for Engaged Learning in Graphics and Visualization Course
• Creativity Development through Inquiry-Based Learning in Biomedical Sciences

Submitted: 31 December 2008 Revised: 14 May 2009 Accepted: 28 May 2009

© 2009 by The American Society for Cell Biology

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Identifying problems and solutions in scientific text

Affiliation.

• 1 Department of Computer Science and Technology, University of Cambridge, 15 JJ Thomson Avenue, Cambridge, CB3 0FD UK.
• PMID: 30147202
• PMCID: PMC6096660
• DOI: 10.1007/s11192-018-2718-6

Research is often described as a problem-solving activity, and as a result, descriptions of problems and solutions are an essential part of the scientific discourse used to describe research activity. We present an automatic classifier that, given a phrase that may or may not be a description of a scientific problem or a solution, makes a binary decision about problemhood and solutionhood of that phrase. We recast the problem as a supervised machine learning problem, define a set of 15 features correlated with the target categories and use several machine learning algorithms on this task. We also create our own corpus of 2000 positive and negative examples of problems and solutions. We find that we can distinguish problems from non-problems with an accuracy of 82.3%, and solutions from non-solutions with an accuracy of 79.7%. Our three most helpful features for the task are syntactic information (POS tags), document and word embeddings.

Keywords: Discourse; Machine learning; Problem-solving patterns.

PubMed Disclaimer

Example of problem-solving pattern when…

Example of problem-solving pattern when applied to the Goldilocks story. Reproduced with permission…

Example of our extraction method…

Example of our extraction method for problems using dependencies. (Color figure online)

Similar articles

• Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition. Jauregi Unanue I, Zare Borzeshi E, Piccardi M. Jauregi Unanue I, et al. J Biomed Inform. 2017 Dec;76:102-109. doi: 10.1016/j.jbi.2017.11.007. Epub 2017 Nov 13. J Biomed Inform. 2017. PMID: 29146561
• Knowledge based word-concept model estimation and refinement for biomedical text mining. Jimeno Yepes A, Berlanga R. Jimeno Yepes A, et al. J Biomed Inform. 2015 Feb;53:300-7. doi: 10.1016/j.jbi.2014.11.015. Epub 2014 Dec 12. J Biomed Inform. 2015. PMID: 25510606
• Automatic query generation using word embeddings for retrieving passages describing experimental methods. Aydın F, Hüsünbeyi ZM, Özgür A. Aydın F, et al. Database (Oxford). 2017 Jan 10;2017:baw166. doi: 10.1093/database/baw166. Print 2017. Database (Oxford). 2017. PMID: 28077568 Free PMC article.
• Experimental study of learning support through examples in mathematical problem posing. Kojima K, Miwa K, Matsui T. Kojima K, et al. Res Pract Technol Enhanc Learn. 2015;10(1):1. doi: 10.1007/s41039-015-0001-5. Epub 2015 Jun 23. Res Pract Technol Enhanc Learn. 2015. PMID: 30613212 Free PMC article.
• A comparative study of supervised learning as applied to acronym expansion in clinical reports. Joshi M, Pakhomov S, Pedersen T, Chute CG. Joshi M, et al. AMIA Annu Symp Proc. 2006;2006:399-403. AMIA Annu Symp Proc. 2006. PMID: 17238371 Free PMC article.
• Coevolution of COVID-19 research and China's policies. Cheng X, Tang L, Zhou M, Wang G. Cheng X, et al. Health Res Policy Syst. 2021 Sep 6;19(1):121. doi: 10.1186/s12961-021-00770-6. Health Res Policy Syst. 2021. PMID: 34488797 Free PMC article.
• Bipartite Network of Interest (BNOI): Extending Co-Word Network with Interest of Researchers Using Sensor Data and Corresponding Applications as an Example. Dai Z, Hu K, Xie J, Shen S, Zheng J, Wu H, Guo Y. Dai Z, et al. Sensors (Basel). 2021 Mar 1;21(5):1668. doi: 10.3390/s21051668. Sensors (Basel). 2021. PMID: 33804324 Free PMC article.
• Baccianella S, Esuli A, Sebastiani F. Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. LREC. 2010;10:2200–2204.
• Becker, M., Palmer, A., & Frank, A. (2016). Clause types and modality in argumentative microtexts. In Workshop on foundations of the language of argumentation (in conjunction with COMMA).
• Briscoe, T., Carroll, J., & Watson, R. (2006). The second release of the rasp system. In Proceedings of the COLING/ACL on interactive presentation sessions, association for computational linguistics pp. 77–80.
• Chandrasekaran B. Towards a taxonomy of problem solving types. AI Magazine. 1983;4(1):9.
• Charles M. Adverbials of result: Phraseology and functions in the problem-solution pattern. Journal of English for Academic Purposes. 2011;10(1):47–60. doi: 10.1016/j.jeap.2011.01.002. - DOI

Related information

Linkout - more resources, full text sources.

• Europe PubMed Central
• PubMed Central

Other Literature Sources

• The Lens - Patent Citations
• scite Smart Citations
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 20 June 2014

The Strength of the Strongest Ties in Collaborative Problem Solving

• Yves-Alexandre de Montjoye 1 ,
• Arkadiusz Stopczynski 1 , 2 ,
• Erez Shmueli 1 ,
• Alex Pentland 1 &
• Sune Lehmann 2 , 3  

Scientific Reports volume  4 , Article number:  5277 ( 2014 ) Cite this article

• Applied mathematics
• Applied physics
• Computational science

Complex problem solving in science, engineering and business has become a highly collaborative endeavor. Teams of scientists or engineers collaborate on projects using their social networks to gather new ideas and feedback. Here we bridge the literature on team performance and information networks by studying teams' problem solving abilities as a function of both their within-team networks and their members' extended networks. We show that, while an assigned team's performance is strongly correlated with its networks of expressive and instrumental ties, only the strongest ties in both networks have an effect on performance. Both networks of strong ties explain more of the variance than other factors, such as measured or self-evaluated technical competencies, or the personalities of the team members. In fact, the inclusion of the network of strong ties renders these factors non-significant in the statistical analysis. Our results have consequences for the organization of teams of scientists, engineers and other knowledge workers tackling today's most complex problems.

Similar content being viewed by others

Interpersonal relationships drive successful team science: an exemplary case-based study

Interplay between success and patterns of human collaboration: case study of a Thai Research Institute

Towards understanding the characteristics of successful and unsuccessful collaborations: a case-based team science study

Introduction.

Complex problems in science, engineering, or business are being solved by teams of people working closely with one another, each with the help of their network. In science, modern experiments require the collaboration and specialization of many individuals 1 . For example, a modern Nature paper can have more than 100 co-authors 2 and the number of co-authors of PNAS papers has more than doubled over the last 20 years, reaching an average of 8.4 co-authors per paper 3 . In businesses, teams of knowledge workers have become the basic unit carrying out work 4 . Our ability to solve complex problems increasingly depends on teams of scientists, engineers, or knowledge workers and their extended information networks 5 , 6 .

Qualitative and quantitative study of high-performing teams–an interdependent collection of individuals working towards a common goal where members share individual and mutual responsibility for the outcome 7 –has been an ongoing effort in the social, management and science of science 8 , 9 , 10 , 11 , 12 , 13 . Previous studies focused on how the personalities, technical or cognitive abilities, or the existence of previous collaborations of team members explain team performance. Recent quantitative studies investigated the determinants of high-performing teams by studying their structure or pattern of communications 14 , 15 , 16 , 17 , 18 , 19 .

Accessing to the right piece of information is central to solving complex problems. This information, however, often only exists in the form of advice, expertise, implicit knowledge, or experience and flows through social ties. Consequently, the structure of social interactions has been shown to enhance or hinder access to such resources. Building on advances in social network analysis, empirical research showed the impact of an individual's information or collaboration network on her performance 20 , 21 , 22 , 23 , 24 , 25 , 26 . Amongst others, the impact of an individual position in the information network has been investigated through measures of node degree, centrality, structural holes, closure and social diversity 27 , 28 , 29 , 30 , 31 , 32 .

Both within-team and extended information networks are useful. Within-team networks allow for engagement, collaboration and the higher level of information sharing needed for teams to perform 15 . Frequent interactions between team members have been shown to help them become familiar with one another and to positively impact their teamwork 33 . Extended networks of informal ties of team members have been shown to be the vector for key exchanges of information 15 . Information often flows through these ties despite the existence of formal coordination and communication mechanisms. These informal extended ties have been shown to be particularly important in competitive environments 34 . This work, at the intersection of information networks and team performance, studies the problem solving abilities of teams as a function of the within-team network structure and extended information network in a real working environment. We show that, for both within team and extended ties networks, only the strongest ties matter.

We examine the performance of 45 assigned teams of four students during one semester. Eighty participants worked in teams on three separate projects for one course. Teams changed for every project so that no one worked with the same person twice. Following the behavioral tradition of organizational learning, we measure performance by focusing on predefined objective outcomes 35 : the grades given by the lecturer to the team reports. Participants specifically mark their individual contribution, which motivates them to actively participate.

There has been considerable ambiguity about what constitutes a tie when studying social and information networks, as well as the structure of organizations 36 , 37 . In his definition of Phillos relationships, Karckhardt 23 focuses on interaction, affection and time as the basic characteristics of ties. This prompted Lincoln to propose to define ties as either instrumental or expressive 17 , 38 . We adopt this definition here.

Networks of instrumental and expressive ties, as well as assigned teams for the first project.

Color saturation, where darker is higher, is the performance of the team and strength of the edges. (A) Directed network of expressive ties of strength four and above. (B) Undirected network of the 20% strongest instrumental ties. Figure 1 shows the network of the strongest expressive and instrumental ties.

Finally, the impact of tie strength on information propagation and knowledge transfer has been discussed in the social network, sociology and science of science literature. It has however been considered in the form of either strong or weak ties, with research showing benefits for both 21 . In his seminal article on job searches, Granovetter showed that opportunities such as information about jobs usually flow through weak ties 22 . Subsequent work showed these distant and infrequent weak ties to be useful in the diffusion of information, advice and ideas 41 , 42 , 43 . However, because they are more accessible or willing to help, strong ties have been shown to be essential to the transfer of complex information and tacit knowledge 23 , 24 , 25 , 31 . In organizational settings, strong ties have been shown to reduce conflicts 44 and to be crucial in dealing with stressful or unusual situations 23 . While quantifying what constitutes a strong and weak tie might be difficult, the effect of the expected benefits as a function of tie strength can be quantified.

Network measures of the strongest ties explain team performance better than other a priori measures, making them not significant most of the time. At the beginning of the experiment, participants took a 6-questions technical competencies test on the topic at hand, filled out a personality questionnaire and were asked to self-evaluate how knowledgeable they were about the topic. The six technical questions were averaged as measured technical competency. The strongest ties in both the expressive and instrumental ties networks explain more of the variance than any of the team competencies considered: the mean or maximum of either the self-evaluated competencies, measured competencies, or personality [ Fig. 1 and Methods]. The maximum self-evaluated and measured technical competencies, or personality, were systematically less explanatory than their mean values. None of the personality measures alone or combined as factors are significant. When taken as the only independent variable in a linear model, both mean self-evaluated ( p = .004) and mean measured technical proficiency ( p = .002) are significant. However, adding mean team degree of the strongest ties to both linear models makes both of them not significant ( p > .05). Mean team degree for expressive ties has p -values of .009 when combined with mean self-evaluated competencies and .014 when combined with mean measured competencies (respectively .043 and .094 for instrumental ties).

Network measures of both expressive and instrumental ties explain performance better than self-evaluated and measured technical competencies, or personality. As a measure of comparison, participants were also asked to fill out a short questionnaire when handing in their projects. In this questionnaire, they were asked how they felt their team performed for this particular project. We expected this a posteriori measure to be the best indicator of team performance as self-assessment has been shown to be a reliable indicator of team performance 46 . It indeed was. However, while a posteriori self-assessed team performance explains nearly half of the variance ( r 2 = .462, p < .001), it is only two times what is explained by our a priori strong ties measures.

Temporal precedence has been often neglected in team performance, collaboration networks and social network research 19 , 47 . Our experimental design allows us to answer the question of temporal precedence between expressive network structure and team performance. Does a dense network structure help a team to perform well or does a performing team create dense networks? In this study, we measured the network of expressive ties before the experiment started. We then assigned participants to teams and we see a positive correlation between the strongest expressive ties and team performance.

Our results hold in both expressive and instrumental networks. Our expressive network is measured through traditional questionnaires, while our instrumental network is measured through sociometric means. We do not however expect the strength of a tie in the two networks to be independent as we are, for example, likely to spend more time with our friends. Fig. 3 shows that, while we indeed spend more time with our strongest expressive ties, the distribution of instrumental strength is still broad. This means that ties in both networks really help explain performance.

As expected, the strength of ties in expressive and instrumental networks is not independent.

Tie strength, a variable often ignored, is in fact crucial for understanding teams' problem solving abilities. In our experiment, the project-based work completed by the participants can be characterized as non-routine and complex. Teams were presented with complex problems which focused on creative thinking and applying gained knowledge in a novel context.

To conclude, these results imply that weak ties between scientists, engineers, or other knowledge workers are unlikely to enhance access to information or to help performance. Very strong ties inside teams and between units or research teams are needed. The problem solving abilities of teams of scientists or knowledge workers tackling today's most complex problems could be greatly improved upon by creating very strong instrumental and expressive ties.

This experiment took place during a full semester course (13 weeks) at a large western university [See Fig. 4 ] 48 . The course was an advanced course, involving work with high-level programming, data modelling and simple machine learning. At the beginning of the course, 80 out of the 95 students agreed to participate in the experiment and filled in an initial questionnaire. During the semester, participants successively worked on 3 projects in teams assigned by us. N = 45 teams containing only participants are analyzed in this paper. Participants also answered right after handing-in their project report how they felt their team performed on a 1 to 5 scale: “How do you think your team did.” Participants were informed that this self-assessed performance would not be seen by the lecturer, nor used in the grading process. The team projects were evaluated by the lecturer taking the difficulty of the assignment into account. The assignments focused on handling and visualizing social data, machine learning and network analysis with students solving coding exercises, answering theoretical questions and analyzing their results. At the university, students are encouraged to and commonly work and solve tasks in teams.

Timeline of the study.

Students agree to participate in the study and fill in the initial questionnaire. They are then assigned to team, complete their assignment and are asked how well they think their team did.

Initial questionnaire

At the beginning of the experiment, we asked participants about their self-reported proficiency on a 1–5 scale (“Would you consider yourself a beginner (1) or an expert (5) for this course?”). We also presented them with 6 technical questions assessing their skills related to the course and their personality (BFI-10) 49 .

Expressive ties

At the beginning of the experiment, we asked participants to rate how well they knew all the other participants. We used the standard grading scale (0–12), which all participants are familiar with: “0 - I do not know this person,” “2 - I recognize this person, but we never talked,” “4 - Acquaintance (we talk or hang out sometimes),” “7 - Friend,” “10 - Close friend,” “12 - One of my best friends”. The list of participants to rate was presented as a webpage that included the participants' full names and university profile pictures.

Instrumental ties

Participants have to login on their devices (smartphones, computers, etc) to use the university wifi system. This allows us to know which building a device was connected to with a resolution of 10 minutes. We infer instrumental ties from the observed co-occurrences using 10 minute time bins. The tie strength S i,j between participants i and j is given by 50 :

We assigned the participants into teams of four, as required by the course lecturer. We assigned students to new teams for each of the three projects so that no student was with the same person in team more than once. To find a balance among within and between team ties, we optimized the entropy of the motifs and the number of edges within teams while adding a penalty for missing motifs using a greedy algorithm [see SM].

The effects of ties of all strength, self-evaluated and measured technical competencies and self-assessed team performance on the performance of teams were evaluated by computing Pearson's product-moment coefficient between the team grade and the mean or the maximum of the measure of interest on a per team basis. The effect of personality was evaluated using both an average of the five personality traits of the team members as well as jointly modeled using a linear model with team grade as dependent variable. Finally, the effect of adding network measures to the self-evaluated and measured technical competencies and personalities of teams members were evaluated using a linear model.

de Solla Price, D. J. Little science, big science… and beyond (Columbia University Press New York, 1986).

Greene, M. The demise of the lone author. Nature 450, 1165–1165 (2007).

Article   CAS   ADS   Google Scholar  

Börner, K., Maru, J. T. & Goldstone, R. L. The simultaneous evolution of author and paper networks. PNAS 101, 5266–5273 (2004).

Article   ADS   Google Scholar  

Beyerlein, M. M., Johnson, D. A. & Beyerlein, S. T. Advances in interdisciplinary studies of work teams: Knowledge work in teams, Vol. 2. (Elsevier Science/JAI Press, 1995).

Wuchty, S., Jones, B. F. & Uzzi, B. The increasing dominance of teams in production of knowledge. Science 316, 1036–1039 (2007).

Lander, E. S. et al. Initial sequencing and analysis of the human genome. Nature 409, 860–921 (2001).

Hackman, J. R. & Oldham, G. R. Work redesign (Addison-Wesley, 1980).

Hong, L. & Page, S. E. Groups of diverse problem solvers can outperform groups of highability problem solvers. PNAS 101, 16385–16389 (2004).

Stokols, D., Hall, K. L., Taylor, B. K. & Moser, R. P. The science of team science: overview of the field and introduction to the supplement. American Journal of Preventive Medicine 35, S77–S89 (2008).

Article   Google Scholar  

Hackman, J. R. The design of work teams. Ariel 129, 32–197 (1987).

Google Scholar  

Hackman, J. R. & Hackman, J. R. Groups that work (and those that don't): Creating conditions for effective teamwork (Jossey-Bass San Francisco, 1990).

Cummings, J. N. & Cross, R. Structural properties of work groups and their consequences for performance. Social Networks 25, 197–210 (2003).

Sanna, L. J. & Parks, C. D. Group research trends in social and organizational psychology: Whatever happened to intragroup research. Psychological Science 8, 261–267 (1997).

Saavedra, S., Hagerty, K. & Uzzi, B. Synchronicity, instant messaging and performance among financial traders. PNAS 108, 5296–5301 (2011).

Pentland, A. The new science of building great teams. Harvard Business Review 90, 60–69 (2012).

Wu, L., Waber, B., Aral, S., Brynjolfsson, E. & Pentland, A. Mining face-to-face interaction networks using sociometric badges: Predicting productivity in an it configuration task. Available at SSRN 1130251 (2008).

Balkundi, P. & Harrison, D. A. Ties, leaders and time in teams: Strong inference about network structures effects on team viability and performance. Academy of Management Journal 49, 49–68 (2006).

Yang, H.-L. & Tang, J.-H. Team structure and team performance in is development: a social network perspective. Information & Management 41, 335–349 (2004).

Guimerà, R., Uzzi, B., Spiro, J. & Amaral, L. A. N. Team assembly mechanisms determine collaboration network structure and team performance. Science 308, 697–702 (2005).

Newman, M. E. The structure of scientific collaboration networks. PNAS 98, 404–409 (2001).

Article   CAS   ADS   MathSciNet   Google Scholar  

Aral, S. & Van Alstyne, M. The diversity-bandwidth trade-off. American Journal of Sociology 117, 90–171 (2011).

Granovetter, M. S. The strength of weak ties. American journal of sociology 1360–1380 (1973).

Krackhardt, D. The strength of strong ties: The importance of philos in organizations. Networks and organizations: Structure, form and action 216, 239 (1992).

Ghoshal, S., Korine, H. & Szulanski, G. Interunit communication in multinational corporations. Management Science 40, 96–110 (1994).

Hansen, M. T. The search-transfer problem: The role of weak ties in sharing knowledge across organization subunits. Administrative science quarterly 44, 82–111 (1999).

Peng, M. W. & Luo, Y. Managerial ties and firm performance in a transition economy: The nature of a micro-macro link. Academy of management journal 43, 486–501 (2000).

Blansky, D. et al. Spread of academic success in a high school social network. PloS one 8, e55944 (2013).

Aral, S., Brynjolfsson, E. & Van Alstyne, M. Information, technology and information worker productivity: Task level evidence. (2007).

Soda, G., Usai, A. & Zaheer, A. Network memory: The influence of past and current networks on performance. Academy of Management Journal 47, 893–906 (2004).

Vaquero, L. M. & Cebrian, M. The rich club phenomenon in the classroom. Sci. Rep. 3 (2013).

Abbasi, A. & Altmann, J. On the correlation between research performance and social network analysis measures applied to research collaboration networks. In: 44th Hawaii International Conference on Systems Science (HICSS-44), Jan. 4–7, Hawaii, USA. (2011).

Aral, S., Brynjolfsson, E. & Van Alstyne, M. Productivity effects of information diffusion in networks. (2007).

Harrison, D. A., Mohammed, S., McGrath, J. E., Florey, A. T. & Vanderstoep, S. W. Time matters in team performance: Effects of member familiarity, entrainment and task discontinuity on speed and quality. Personnel Psychology 56, 633–669 (2003).

Ancona, D. G. Outward bound: strategic for team survival in an organization. Academy of Management Journal 33, 334–365 (1990).

Article   MathSciNet   Google Scholar  

Levitt, B. & March, J. G. Organizational learning. Annual Review of Sociology 319–340 (1988).

Wuchty, S. What is a social tie? PNAS 106, 15099–15100 (2009).

Kilduff, M. & Tsai, W. Social networks and organizations (Sage, 2003).

Lincoln, J. R. & Miller, J. Work and friendship ties in organizations: A comparative analysis of relation networks. Administrative Science Quarterly 181–199 (1979).

Waber, B. N. et al. Organizational engineering using sociometric badges. In: International Conference on Network Science, New York, NY (2007).

Eagle, N., Pentland, A. S. & Lazer, D. Inferring friendship network structure by using mobile phone data. PNAS 106, 15274–15278 (2009).

Rogers, E. M. Diffusion of innovations (Simon and Schuster, 2010).

Uzzi, B. & Lancaster, R. Relational embeddedness and learning: The case of bank loan managers and their clients. Management science 49, 383–399 (2003).

Constant, D., Sproull, L. & Kiesler, S. The kindness of strangers: The usefulness of electronic weak ties for technical advice. Organization science 7, 119–135 (1996).

Nelson, R. E. The strength of strong ties: Social networks and intergroup conflict in organizations. Academy of Management Journal 32, 377–401 (1989).

Cronbach, L. J. Coefficient alpha and the internal structure of tests. Psychometrika 16, 297–334 (1951).

Falchikov, N. Product comparisons and process benefits of collaborative peer group and self assessments. Assessment and Evaluation in Higher Education 11, 146–166 (1986).

Kozlowski, S. W. & Bell, B. S. Work groups and teams in organizations. Handbook of psychology (2003).

Stopczynski, A. et al. Measuring large-scale social networks with high resolution. PLoS ONE 9 (2014).

Rammstedt, B. & John, O. P. Measuring personality in one minute or less: A 10-item short version of the big five inventory in english and german. Journal of Research in Personality 41, 203–212 (2007).

Sapiezynski, P. Measuring patterns of human behavior through fixed-location sensors. (2013). URL www2.imm.dtu.dk/pubdb/views/edoc_download.php/6556/pdf/imm6556.pdf . (Date of access: 27/03/2014).

Download references

Acknowledgements

We thank Piotr Sapiezynski for his help in data collection and insights regarding wifi data, Vedran Sekara for his help in data collection and Luc Rocher for his help with the network visualizations. We also thank the students for participating in the experiment. Yves-Alexandre de Montjoye thanks the Belgian American Educational Foundation (B.A.E.F.) for its financial support. This research was partially sponsored by the Army Research Laboratory under Cooperative Agreement Number W911NF-09-2-0053. Views and conclusions in this document are those of the authors and should not be interpreted as representing the policies, either expressed or implied, of the Army Research Laboratory.

Author information

Authors and affiliations.

Media Lab, Massachusetts Institute of Technology,

Yves-Alexandre de Montjoye, Arkadiusz Stopczynski, Erez Shmueli & Alex Pentland

Department of Applied Mathematics and Computer Science, Technical University of Denmark,

Arkadiusz Stopczynski & Sune Lehmann

Niels Bohr Institute, University of Copenhagen,

Sune Lehmann

You can also search for this author in PubMed   Google Scholar

Contributions

Y.-A. de M. designed the experiment, analyzed the data and wrote the paper; A.S. and E.S. designed the experiment and analyzed the data; A.P. and S.L. designed the experiment and wrote the paper.

Ethics declarations

Competing interests.

The authors declare no competing financial interests.

Electronic supplementary material

Supplementary information.

Supplementary Information for 'The Strength of the Strongest Ties in Collaborative Problem Solving'

Rights and permissions

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder in order to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/

Reprints and permissions

About this article

Cite this article.

de Montjoye, YA., Stopczynski, A., Shmueli, E. et al. The Strength of the Strongest Ties in Collaborative Problem Solving. Sci Rep 4 , 5277 (2014). https://doi.org/10.1038/srep05277

Download citation

Received : 28 March 2014

Accepted : 14 May 2014

Published : 20 June 2014

DOI : https://doi.org/10.1038/srep05277

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Inclusion unlocks the creative potential of gender diversity in teams.

• Balázs Vedres
• Orsolya Vásárhelyi

Scientific Reports (2023)

Research collaboration networks in maturing academic environments

• Luís Filipe de Miranda Grochocki
• Andrea Felippe Cabello

Scientometrics (2023)

Social ties influence teamwork when managing clinical emergencies

• Maria B. Rasmussen
• Martin G. Tolsgaard
• Charlotte V. Ringsted

BMC Medical Education (2020)

Social ties between team members affect patient satisfaction: a data-driven approach to handling complex network analyses

• Betina Ristorp Andersen
• Jesper Løve Hinrich

Advances in Health Sciences Education (2020)

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Starting the research process
• How to Write a Problem Statement | Guide & Examples

How to Write a Problem Statement | Guide & Examples

Published on November 6, 2022 by Shona McCombes and Tegan George. Revised on November 20, 2023.

A problem statement is a concise and concrete summary of the research problem you seek to address. It should:

• Contextualize the problem. What do we already know?
• Describe the exact issue your research will address. What do we still need to know?
• Show the relevance of the problem. Why do we need to know more about this?
• Set the objectives of the research. What will you do to find out more?

Table of contents

When should you write a problem statement, step 1: contextualize the problem, step 2: show why it matters, step 3: set your aims and objectives.

Problem statement example

Other interesting articles

Frequently asked questions about problem statements.

There are various situations in which you might have to write a problem statement.

In the business world, writing a problem statement is often the first step in kicking off an improvement project. In this case, the problem statement is usually a stand-alone document.

In academic research, writing a problem statement can help you contextualize and understand the significance of your research problem. It is often several paragraphs long, and serves as the basis for your research proposal . Alternatively, it can be condensed into just a few sentences in your introduction .

A problem statement looks different depending on whether you’re dealing with a practical, real-world problem or a theoretical issue. Regardless, all problem statements follow a similar process.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

The problem statement should frame your research problem, giving some background on what is already known.

Practical research problems

For practical research, focus on the concrete details of the situation:

• Where and when does the problem arise?
• Who does the problem affect?
• What attempts have been made to solve the problem?

Theoretical research problems

For theoretical research, think about the scientific, social, geographical and/or historical background:

• What is already known about the problem?
• Is the problem limited to a certain time period or geographical area?
• How has the problem been defined and debated in the scholarly literature?

The problem statement should also address the relevance of the research. Why is it important that the problem is addressed?

Don’t worry, this doesn’t mean you have to do something groundbreaking or world-changing. It’s more important that the problem is researchable, feasible, and clearly addresses a relevant issue in your field.

Practical research is directly relevant to a specific problem that affects an organization, institution, social group, or society more broadly. To make it clear why your research problem matters, you can ask yourself:

• What will happen if the problem is not solved?
• Who will feel the consequences?
• Does the problem have wider relevance? Are similar issues found in other contexts?

Sometimes theoretical issues have clear practical consequences, but sometimes their relevance is less immediately obvious. To identify why the problem matters, ask:

• How will resolving the problem advance understanding of the topic?
• What benefits will it have for future research?
• Does the problem have direct or indirect consequences for society?

Finally, the problem statement should frame how you intend to address the problem. Your goal here should not be to find a conclusive solution, but rather to propose more effective approaches to tackling or understanding it.

The research aim is the overall purpose of your research. It is generally written in the infinitive form:

• The aim of this study is to determine …
• This project aims to explore …
• This research aims to investigate …

The research objectives are the concrete steps you will take to achieve the aim:

• Qualitative methods will be used to identify …
• This work will use surveys to collect …
• Using statistical analysis, the research will measure …

The aims and objectives should lead directly to your research questions.

Learn how to formulate research questions

Prevent plagiarism. Run a free check.

You can use these steps to write your own problem statement, like the example below.

Step 1: Contextualize the problem A family-owned shoe manufacturer has been in business in New England for several generations, employing thousands of local workers in a variety of roles, from assembly to supply-chain to customer service and retail. Employee tenure in the past always had an upward trend, with the average employee staying at the company for 10+ years. However, in the past decade, the trend has reversed, with some employees lasting only a few months, and others leaving abruptly after many years.

Step 2: Show why it matters As the perceived loyalty of their employees has long been a source of pride for the company, they employed an outside consultant firm to see why there was so much turnover. The firm focused on the new hires, concluding that a rival shoe company located in the next town offered higher hourly wages and better “perks”, such as pizza parties. They claimed this was what was leading employees to switch. However, to gain a fuller understanding of why the turnover persists even after the consultant study, in-depth qualitative research focused on long-term employees is also needed. Focusing on why established workers leave can help develop a more telling reason why turnover is so high, rather than just due to salaries. It can also potentially identify points of change or conflict in the company’s culture that may cause workers to leave.

Step 3: Set your aims and objectives This project aims to better understand why established workers choose to leave the company. Qualitative methods such as surveys and interviews will be conducted comparing the views of those who have worked 10+ years at the company and chose to stay, compared with those who chose to leave.

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

Methodology

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

Once you’ve decided on your research objectives , you need to explain them in your paper, at the end of your problem statement .

Keep your research objectives clear and concise, and use appropriate verbs to accurately convey the work that you will carry out for each one.

I will compare …

All research questions should be:

• Focused on a single problem or issue
• Researchable using primary and/or secondary sources
• Feasible to answer within the timeframe and practical constraints
• Specific enough to answer thoroughly
• Complex enough to develop the answer over the space of a paper or thesis
• Relevant to your field of study and/or society more broadly

Research objectives describe what you intend your research project to accomplish.

They summarize the approach and purpose of the project and help to focus your research.

Your objectives should appear in the introduction of your research paper , at the end of your problem statement .

Your research objectives indicate how you’ll try to address your research problem and should be specific:

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. & George, T. (2023, November 20). How to Write a Problem Statement | Guide & Examples. Scribbr. Retrieved August 15, 2024, from https://www.scribbr.com/research-process/problem-statement/

Is this article helpful?

Shona McCombes

Other students also liked, how to choose a dissertation topic | 8 steps to follow, how to define a research problem | ideas & examples, writing strong research questions | criteria & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• Research Process
• Manuscript Preparation
• Manuscript Review
• Publication Process
• Publication Recognition
• Language Editing Services
• Translation Services

How to Write an Effective Problem Statement for Your Research Paper

• 4 minute read

Table of Contents

The problem statement usually appears at the beginning of an article, making it one of the first things readers encounter. An excellent problem statement not only explains the relevance and importance of the research but also helps readers quickly determine if the article aligns with their interests by clearly defining the topic. Therefore, the problem statement plays a unique role in the widespread dissemination of the paper and enhancing the researcher’s academic influence.  

In this article, we will focus on writing ideas, structure, and practical examples of the problem statement, helping researchers easily write an excellent problem statement.  

Basic Writing Strategies for the Problem Statement  

The problem statement aims to highlight the pressing issue the research intends to address. It should be concise and to the point. Researchers can follow a two-step approach: first, think about the content of the problem statement, and then organize the writing framework.  

Before writing, clarify the following points¹ :  

• What is the reader’s level of understanding of the research topic?  
• How can the significance of the research be effectively conveyed to the reader?  

After addressing these two questions, you can organize the content according to the following structure:  

• Clarify what you aim to achieve with your research.  
• Explore why the problem exists and explain how solving it helps reach the goal.  
• Outline the potential impact of the research, such as possible outcomes, challenges, and benefits.  
• Recommend a plan for your experiment that follows the rules of science.  
• Explain the potential consequences if the problem is not resolved (if applicable).  

Three Important Parts of the Problem Statement  

The content and length of the problem statement can vary depending on the type of research. Although there’s no fixed format, it’s helpful to include these three key parts:  

  Research Background:  

Explain clearly what problem your research focuses on. Describe how things would be better if this problem didn’t exist. Also, talk about what other researchers have tried to do about this problem and what still needs to be figured out.  

  Research Significance:  

Clarify the impact of the problem on the research field and society, and analyze the cause of the problem. Explain who will benefit from solving the problem, thus demonstrating the relevance of the research and its contribution to the existing research system.²  To illustrate the relevance, consider aspects such as the geographical location or process where the problem occurs, the time period during which it exists, and the severity of the problem.  

Solution:  

Describe the research objective and the expected solution or results.  

Understanding the Writing Method Through Examples  

To further explore the writing method of the problem statement, let’s look at the following case.  

Research Topic: 

The benefits of vitamin D supplementation on the immune system.  

Problem Statement: 

• Review existing research on the role of vitamin D in the immune system, emphasizing the potential impacts of vitamin D deficiency on the human body.  
• List the obstacles encountered when trying to increase vitamin D levels in the body through supplements, and briefly mention the physiological or molecular mechanisms behind these obstacles.  
• Clarify feasible ways to overcome these obstacles, such as new methods to promote the absorption of vitamin D in the intestine. Then, focus on the benefits of these methods, such as helping postmenopausal women with breast cancer improve their blood vitamin D levels.   

Points to Note: 

When crafting your problem statement, focus on essential details and avoid unnecessary information. Additionally, absolute terms such as “must” should be avoided.  

( The examples in this article are used only to illustrate writing points, and the academic views contained therein are not for reference. )  

By mastering these techniques and methods, you can enhance the clarity and impact of their problem statements. This not only makes the articles more engaging for reviewers and readers but also increases the likelihood of broader dissemination.  

For efficient and professional assistance, consider reaching out to Elsevier Language Services. Our team of expert editors, who are native English speakers across various disciplines, can help refine every aspect of your article, including the problem statement. Our goal is to ensure your research achieves efficient publication and has wide-reaching impact, supporting your academic journey in the long term.  

Type in wordcount for Plus Total: USD EUR JPY Follow this link if your manuscript is longer than 9,000 words. Upload

References:  

• SURF Workshop Resources: Problem Statements – Purdue OWL® – Purdue University. (n.d.). https://owl.purdue.edu/owl/subject_specific_writing/writing_in_the_purdue_surf_program/surf_workshop_resources_problem_statements/index.html
• Problem Statement | A practical guide to delivering results. (n.d.). Copyright (C)2024 a Practical Guide to Delivering Results. All Rights Reserved. https://deliveringresults.leeds.ac.uk/delivering-results-lifecycle/problem-statement/

What is and How to Write a Good Hypothesis in Research?

How to Use Tables and Figures effectively in Research Papers

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

The Value of Openness in Scientific Problem Solving

• Opening up problem information to a large group of outsiders can yield innovative technical solutions, increase the probability of success in science programs, and ultimately boost research productivity.
• Open source software communities provide a model for improving the process of solving scientific problems.
• Outsiders can see problems with fresh eyes; in this study, problems were solved by independent scientists with expertise at the boundary of or even outside their field.
• Achieving true openness and collaboration will require change in the mindsets of both scientists and lab leadership.

Author Abstract

Openness and free information sharing amongst scientists are supposed to be core norms of the scientific community. However, many studies have shown that these norms are not universally followed. Lack of openness and transparency means that scientific problem solving is constrained to a few scientists who work in secret and who typically fail to leverage the entire accumulation of scientific knowledge available. We present evidence of the efficacy of problem solving when disclosing problem information. The method's application to 166 discrete scientific problems from the research laboratories of 26 firms is illustrated. Problems were disclosed to over 80,000 independent scientists from over 150 countries. We show that disclosure of problem information to a large group of outside solvers is an effective means of solving scientific problems. The approach solved one-third of a sample of problems that large and well-known R&D-intensive firms had been unsuccessful in solving internally. Problem-solving success was found to be associated with the ability to attract specialized solvers with range of diverse scientific interests. Furthermore, successful solvers solved problems at the boundary or outside of their fields of expertise, indicating a transfer of knowledge from one field to others.

Paper Information

• Full Working Paper Text
• Working Paper Publication Date: January 2007
• HBS Working Paper Number: 07-050
• Faculty Unit(s): Technology and Operations Management
• 25 Jun 2024
• Research & Ideas

Rapport: The Hidden Advantage That Women Managers Bring to Teams

• 11 Jun 2024
• In Practice

The Harvard Business School Faculty Summer Reader 2024

How transparency sped innovation in a $13 billion wireless sector.

• 24 Jan 2024

Why Boeing’s Problems with the 737 MAX Began More Than 25 Years Ago

• 27 Jun 2016

These Management Practices, Like Certain Technologies, Boost Company Performance

• Innovation and Management
• Research and Development

Sign up for our weekly newsletter

Mathematical problem-solving in scientific practice

• Original Research
• Open access
• Published: 23 September 2021
• Volume 199 , pages 13621–13641, ( 2021 )

Cite this article

You have full access to this open access article

• Davide Rizza   ORCID: orcid.org/0000-0002-1375-371X 1  

1895 Accesses

Explore all metrics

In this paper I study the activity of mathematical problem-solving in scientific practice, focussing on enquiries in mathematical social science. I identify three salient phases of mathematical problem-solving and adopt them as a reference frame to investigate aspects of applications that have not yet received extensive attention in the philosophical literature.

Similar content being viewed by others

An Empirical Approach to the Mathematical Values of Problem Choice and Argumentation

Issues Regarding the Concept of Mathematical Practices

Philosophy or Not? The Study of Cultures and Practices of Mathematics

Avoid common mistakes on your manuscript.

1 Mathematical resources and problem-solving

The last twenty years have seen a remarkable increase in philosophical studies devoted to the role played by mathematics in scientific practice. Work in this area has, at the same time, covered much ground and opened up opportunities for lines of investigation not yet pursued. For instance, the vast amount of work devoted to modelling with differential or integral equations and computer simulations Footnote 1 contrastively points to areas that have not yet received much attention: one of them, to be discussed in the following sections, is structural modelling in mathematical social science. Footnote 2

This area of research is of philosophical interest because its examination can shed much light, in an especially accessible and illuminating way, on specific aspects of the application of mathematics that are recognised to matter but have not been widely studied so far. The aspect of applications I shall be focussing on in this paper has been identified in the following terms by Ardourel, Barberousse and Imbert:

When analyzing scientific representations, philosophers of science are keen on mentioning that some models provide scientists with “mathematical resources” and “inferential power”, but they seldom give a detailed analysis of these notions. (Ardourel et al. 2018 , p. 2)

The authors of Ardourel et al. ( 2018 ) provide part of the missing analysis for a specific family of resources, namely mathematical formalisms. They also note that structures and the mathematical results associated with them constitute natural mathematical resources but do not devote their investigation to them. It is the goal of this paper to do so through a study of significant episodes in mathematical social science, more specifically mathematical voting theory.

In order to understand the role of structural resources within scientific practice, it seems to me especially helpful to look at them in connection with problem-solving. Something functions as a resource if it can support a particular activity. Thus, being a resource is best spelled out in connection with a problem-solving activity. Structures or structured objects play the role of resources if they help tackle problems within a field of enquiry. When that happens, the generic qualification of “inferential power” associated with their employment can be more informatively reduced to the effectiveness of the methods erected upon them.

In order to provide a sufficiently thorough philosophical elaboration of the last remarks, it is helpful to identify certain turning points within scientific problem-solving, in which structural objects play an important role. I refer to these stages of enquiry simply as phase (a), phase (b) and phase (c). In outline, phase (a) is a stage at which the terms of certain problems under investigation are assigned formal characters in such a way that the problems themselves can be referred to a formal working environment. Phase (b) is a stage at which the given environment is used to introduce problem-solving techniques. Phase (c) is the stage at which the formal working environment from phase (a) is modified to assimilate new problems and the store of techniques from phase (b) is expanded to deal with such new problems. I shall refer to phases (a) to (c) to articulate the analysis of concrete problem-solving episodes in scientific practice. A further clarification of what these phases amount to can only be offered through the examination of relevant enquiries, which occupies the rest of the paper.

In particular, Sect.  2 provides a more detailed account of phases (a) to (c) in terms of a quick, preliminary illustration. Sections  3 and  4 , on the other hand, offer a progressively broader account of these phases in the context of more extensive problem-solving activities.

In the remainder of the present section I only wish to show how (a) to (c) can be rather naturally singled out in a contrastive manner, by looking at aspects of the application of mathematics that have been neglected by recent, influential lines of investigation.

Over the last few decades, many contributions to philosophy of mathematics have portrayed applications in terms of a correspondence between an empirical and a mathematical structure. Two frequently cited sources are Field ( 1980 ), p. 6 and Bueno and Colyvan ( 2011 ), pp. 346–347 Footnote 3 ). The latter article recognises that the world does not come equipped with a set of objects [...] and sets of relations on those’ (Bueno and Colyvan 2011 , p. 347), i.e. as a cluster of structured settings. Nonetheless, the problem of examining how structured settings are arrived at is not raised or pursued. The decision to neglect this issue may be due to a remark on p. 354, to the effect that empirical settings are not required to be given a mathematics-free description. The remark is not unreasonable Footnote 4 but it shifts attention away from the fact that it is an important phase of scientific investigations to introduce a structured presentation of relevant contents that supports the progress of enquiry. The introduction of such a presentation is phase (a).

The stress of Bueno and Colyvan ( 2011 ) on the world in the quote from 1 above is, in my opinion, more fruitfully replaced by a stress on problems. The fact that scientific enquiry is a human activity endows it with a distinctive epistemological status. It is a confrontation between an agent and her surroundings in which means have to be devised in order to make those surroundings decipherable and to enable specific interventions upon them. What is pressing, under such circumstances, is the resolution of problems. Desired information is to be gained and specific goals (e.g. predictive or technological) are to be achieved. Thus, if phase (a) is to support the progress of enquiry, it must do so by introducing a structured environment in which it is possible to develop problem-solving techniques. Their construction is carried out in phase (b).

An independent motivation for phase (b), as well as a motivation for phase (c), may be spelled out by considering a recent topic of dominant philosophical interest in connection with the application of mathematics, i.e. mathematical explanation. Contributions to the debates on mathematical explanation look at scenarios in which mathematical resources are already available to account for certain given facts. Such scenarios, especially as they are connected to the problems of clarifying whether or not a mathematical result has explanatory power, exclude an interest in the work done before the resources used to provide explanations were available. This work is the construction of problem-solving techniques, i.e. phase (b). Moreover, work on mathematical explanation (as important articles on this topic like Lyon ( 2012 ), Lange ( 2013 ), Lange ( 2017 ) show) has typically restricted attention to the use of individual mathematical results to explain individual phenomena. Specialised attention to contexts of the latter kind naturally leads to a neglect of broader ones, in which not only many interrelated problems are initially present, but systematic attempts to relate them to new problems is made. When these attempts are made the subject of specific analysis, phase (c) emerges.

It is worth clarifying that my goal, when making the previous points, was not to criticise existing approaches, with a view to, say, rejecting them, but simply to rely upon them to identify neglected aspects of applications. In this connection, I do not deny that many current, wide-ranging studies of applications are sensitive to the aspects of applications I am drawing attention to. A dedicated study of precisely these aspects is, however, still missing and my aim is to supply it (or to begin to supply it).

For instance, the interest in classifying the epistemic contributions of mathematics to scientific enquiry, which drives extensive analyses in Pincock ( 2012 ), falls in line with the aims of this paper. The epistemic contributions I focus on are not, however, part of Pincock’s discussion, largely because of its emphasis on representation, as opposed to problem-solving.

More generally, I am hoping to provide, in what follows, a study of applications from a viewpoint that can constructively enrich several ongoing lines of investigation by exploring areas that they have been neglected (as in the case of Bueno and Colyvan ( 2011 ) or Pincock ( 2012 )) or by extending their purview beyond issues they have already applied themselves to (as in the case of Ardourel et al. ( 2018 )).

2 Phases of problem-solving

My goal in this section is to offer a more concrete impression of what phases (a) to (c) are, before studying them in greater detail and within broader contexts. It is important to recall that, in general, phases (a) to (c) emerge because problems that currently resist solution have arisen. Phase (b) introduces mathematical techniques designed to tackle them. Their introduction is, in structural modelling, made possible by the assignment of formal characters to the terms of the given problems. Only if these problems’ terms can be regarded in reasoning as particular mathematical items can mathematical techniques be devised to act upon them. This is why problems have to be set within a suitable mathematical environment before they can be tackled. Phase (a) sets them in the environment that supports phase (b).

In general, the execution of phases (a) and (b) does not merely lead to the resolution of pre-existing problems. It also produces a richer conception of these problems, refining existing ones and relating them to new problems. The environment originally set up in phase (a) may prove inadequate to deal with some new problems: when this is the case, phase (c) takes place. The original environment is modified through the addition of new features or the integration into a more encompassing environment, from which new problem-solving techniques emerge.

As a miniature illustration of phases (a), (b) and (c), I now wish to consider a simple practical problem. The problem is to identify published books by numerical codewords for the purpose of ordering, sales reporting and inventory control. Useful codewords enable the detection of errors. For instance, it should be possible to tell if a printing error swapped two digits of a codeword. Footnote 5 This is a highly circumscribed problem belonging to the vastly larger domain of coding theory.

Fixing the length of a codeword at ten digits, it is useful to let the numerical digits denote the elements of a structured object, the formal working environment. An obvious choice for the set of codeword digits is \(\{0, 1, \ldots 9\}\) , but, in order to supply an environment on which addition and multiplication are well-behaved, it is better to adopt \(\{0, 1, \ldots , 10\}\) . The latter set, which I shall refer to as \(Z_{11}\) , can be endowed with canonical field structure. Footnote 6

Phase (a) amounts to the introduction of \(Z_{11}\) for the purpose of carrying out phase (b), i.e. the design of an error-detecting code. Footnote 7 One possibility is to let codewords be of the form \(x_{1}, \ldots , x_{10}\) , where the first nine digits \(x_{1}, \ldots , x_{9}\) identify a book, its publisher and its language, as desired, while \(x_{10}\) , the tenth digit, Footnote 8 satisfies the following equality modulo 11:

The above modular equality entails that its sides are both multiples of 11. Field arithmetic on the environment \(Z_{11}\) now guarantees the sought error-detecting feature. Swapping two digits in a codeword produces a string that is not a codeword (does not satisfy the last equality). To see how, let a swap of \(x_{m}, x_{n}\) in the codeword \(x_{1}, \ldots , x_{10}\) yield the word \(y_{1},\ldots , y_{10}\) . Then:

and \(y_{1}, \ldots , y_{10}\) is not a codeword because, in \(Z_{11}\) , the product of two nonzero elements must be nonzero. For the same reason, single incorrect digits are detected. Thus, the algebraic structure of the environment \(Z_{11}\) enables error-detection features and helps set up a code that solves the given problem. Footnote 9 The construction of a code, obtained by defining codewords in terms of a relation on \(Z_{11}\) is phase (b). Longer codes with a prime number of digits and an error-detecting features can be constructed along similar lines, using prime fields other than \(Z_{11}\) .

The simple code construction described here highlights the usefulness of finite fields as formal working environments. Phase (c) ensues when other coding problems are taken into account. For instance, problems in cryptography require the introduction of linear codes, studied within the formal working environment of vector spaces over finite fields.

The quick sketch just offered can help paint a more concrete picture of phases (a) to (c), but only in a very limited form. The environment in phase (a) is not always a structure in the model-theoretic sense. Phase (b) does not always reduce to the solution of a single problem: in general, its goal is to tackle, refine and extend a family of initially given problems. Phase (c) is not always a familiar integration of structures but may require finding manageable ways of enriching an environment or amalgamating distinct environments.

The additional dimensions of phases (a) to (c) that could not be taken into account here are explored in the next two sections, which consider progressively more extensive contexts of enquiry within mathematical voting theory. This research area is of special interest because important advances in it have arisen from the construction of geometric working environments. These environments, in turn, have supported the introduction of ingenuous problem-solving techniques.

3 A simple geometric environment

Voting, a collective decision method, relies upon a procedure to associate inputs from voters with an outcome, which may be the choice of an action, a candidate or a policy. It has long been known that voting procedures give rise to undesired or puzzling results. A famous example, originally described by Condorcet in Condorcet ( 1785 ), pp. xvi–xviii, involves nine voters and three candidates A ,  B ,  C . Four voters express the ranking \(A> B > C\) (where the inequality symbol denotes strict subjective preference), three express the ranking \(B> C > A\) and two express the ranking \(C> A > B\) .

The rule of pairwise majority, combining together the outcomes of three sub-elections on the respective pairs ( A ,  B ), ( B ,  C ) and ( C ,  A ), produces the cyclic outcome \(A > B\) , \(B > C\) , \(C > A\) , from which it is impossible to select a winner. The study of cyclic outcomes has been a major theme in modern voting theory since at least Arrow’s theorem (see Arrow 1951 ). Condorcet’s approach, which focusses on a special procedure (i.e. pairwise majority), has been replaced and generalised by the study of conditions that force any procedure satisfying them to produce Condorcet-type situations, in which certain transitive rankings yield a cyclic outcome.

In this context, a voting procedure is usually represented in set-theoretic terms. Candidates or alternatives are regarded as a finite, nonempty set S (with at least three elements). Given n voters, if L ( S ) is the set of linear orders on S , an element of \(L(S)^{n}\) is a list of n strict preferences or a preference profile and a voting procedure may be abstractly regarded as a function \(f: L(S)^{n} \longrightarrow L(S)\) . The problematic occurrence of cyclic outcomes is often translated into the statement of conditions that cannot be satisfied by any function f of the type just described.

Investigations into the unsatisfiability of conditions that force cyclic outcomes is of interest here because it supplies a formal context for the position of problems that cannot be satisfactorily tackled without the introduction of a suitable working environment. This is apparent from a classic contribution to social choice theory (i.e. Sen 1970 ), due to Amartya Sen. Sen observed that the following conditions cannot be simultaneously satisfied by any function from \(L(S)^{n}\) to L ( S ):

at least two voters are decisive on two respective, distinct pairs of alternatives (i.e. the way they rank those pairs is the way they are ranked by the voting procedure);

if every voter exhibits the same ranking of a given pair of alternatives, this is the procedure’s ranking;

Conditions 1 and 2, known as Minimal Liberalism and the Pareto condition respectively, always force some cyclic outcome. To establish his result, Sen followed Condorcet’s approach and described a simple, two-voter scenario in which one voter, decisive on the pair ( A ,  B ), expresses the ranking \(C> A > B\) , while a second voter, decisive on the pair ( B ,  C ), expresses the ranking \(B> C > A\) . The pairwise outcome of the profile just described is a cycle.

Sen’s cyclic scenario raises one interesting issue. Its cyclic outcome depends only on the pairwise rankings \(C > A\) , \(A > B\) from one voter and only on the pairwise rankings \(C> A, B > C\) from the other voter. These rankings can be regarded indifferently as sub-rankings of transitive or of cyclic rankings. Thus, Sen’s specific example does not make use of the assumption that a procedure should take profiles of transitive rankings as arguments. The problem of determining whether this is always the case when cyclic outcomes occur naturally arises.

This problem is important because a solution shows whether or not Sen’s conditions are sensitive to the transitive behaviour of voters. The desired solution requires a comprehensive understanding of the manner in which Sen’s conditions 1 and 2 operate. Specific cyclic examples do not provide a comprehensive picture. All possible cyclic outcomes forced by conditions 1 and 2 must be taken into account in order to determine whether each one of them can be supported by transitive rankings. A priori , it cannot be ruled out that there may be cyclic outcomes not supported by transitive rankings.

A general analysis of the cycles determined by conditions 1 and 2 requires combinatorial considerations: it essentially concerns the way pairwise rankings of alternatives in a profile are connected to outcomes under the action of conditions 1 and 2. The combinatorial connection is not apparent. An environment is needed to organise the arbitrary combinations of pairwise rankings into a structured object that supports techniques of analysis. The needed organisation is achieved in phase (a).

For the sake of clarity, I illustrate phase (a), as carried out in Li and Saari ( 2008 ), only with respect to three alternatives A ,  B ,  C (the same approach generalises to any finite number of alternatives). The formal working environment is set up through the assignment of two endpoints on a unit segment to the two strict rankings of ( A ,  B ). The strict rankings of ( B ,  C ), ( C ,  A ) are similarly associated with endpoints of other unit segments. When these segments are located on the axes of an orthogonal system, as in Fig.  1 , the unit cube emerges as a structured object supporting the analysis of conditions 1 and 2. The cube’s vertices carry ranking information: for instance, vertex 8 from Fig.  1 codifies the pairwise rankings \(B> A, A> C, C > B\) . Moving away from the origin along one axis reverses exactly one of these rankings. Note that vertices 7, 8 codify cycles and that each edge emanating from a cycle is incident on a transitive vertex (a vertex codifying a transitive ranking).

Pairwise comparison environment

The formal working environment just instituted supports phase (b) for two reasons: (i) it allows a simultaneous geometric representation of profiles and outcomes; (ii) because of (i), it provides a technique to study pairwise comparisons by working with the incidence geometry of the unit cube (hypercubes support similar analyses for more than three alternatives Footnote 10 and may be regarded as an expansion of the formal working environment just described).

Because of (i) and (ii), a geometric technique can be developed to study all cyclic outcomes. This technique relies on the simple fact that conditions 1 and 2 are, in the geometric environment, ways of selecting cube faces. For instance, whenever the Pareto condition applies, the rankings of every agent must lie on the same face (e.g. the pairwise outcome \(A > C\) requires the bottom face in Fig.  1 ). If, in addition, an agent is decisive, her ranking is restricted to a specified edge on that face. No further restrictions ensue, because no agent is decisive on more than one pair.

Thus, profiles supporting a given outcome (a vertex) lie on certain edges of a cube’s face. It follows that every cycle is supported by transitive profiles. This is because the cyclic vertices 7, 8 are linked to transitive vertices in each direction. To see why this matters, consider the cyclic outcome 8. The Pareto condition can be invoked to select the cube’s bottom face, on which 8 lies. The rankings of decisive agents must be taken on the edges emanating from 8, so they are the transitive rankings of vertices 2 and 6. Finally, the transitive ranking of any other agent can be chosen from the transitive vertices on the bottom face of the cube (i.e. 1, 2, 6).

The last considerations yield a uniform method to build all examples of cycles produced by Sen’s conditions. It suffices to select a cyclic vertex and consider the profile in which every voter expresses the same cycle: condition 1 allows moves of voters preferences along edges incident on the cyclic vertex, thus producing the same cyclic outcome on the basis of transitive rankings.

It is worth noting that the technique just described works beyond the three-dimensional case, for a simple reason. A decision outcome lies on the intersection of all faces selected by conditions 1 and 2, whereas an individual ranking lies on the intersection of some faces, which includes the intersection of all faces. Thus, a cyclic outcome is always supported by unanimous cyclic rankings and these rankings can always be turned into transitive ones with the same outcome, using condition 1 to visit adjacent edges or faces (allowing for the added freedom supplied by hypercubes).

Sen’s conditions alone do not provide information on whether there are cyclic outcomes supported only by transitive profiles or cyclic outcomes that are never supported by unanimously cyclic profiles Footnote 11 . The geometric technique developed in phase (b) makes use of the unit cube to obtain the missing information. Moreover, because this technique operates on disconnected pairwise rankings, using only the incidence geometry of the unit cube, it never relies on the transitivity of voters’ rankings. It follows that Sen’s conditions 1 and 2 operate uniformly irrespective of whether or not voter behaviour satisfies transitivity.

It should now be clear that phase (b) provides a way of engaging with conditions 1 and 2 that supports systematic analyses (e.g. by providing a uniform method to generate cyclic outcomes from transitive profiles) and solves the problem posed by the fact that Sen’s example disregards the transitivity of individual rankings. The disregard is not accidental: it accompanies every cyclic outcome constrained by conditions 1 and 2.

The analysis supported by the geometric technique from phase (b) can be further refined by one form of phase (c), namely the addition of structural features to the geometric environment. A simple but insightful instance of phase (c) depends on the addition of topological structure to the faces of the cube, now regarded as a subset of \(\mathbb {R}^{3}\) , as opposed to a point lattice.

The idea is to focus on a specific face F (singled out by the Pareto condition) and take the points of F to codify the ranking of the vertex closest to them or a tie (in case they lie on the midpoint of an edge or on a line joining two midpoints and parallel to an edge). The notion of closeness now in place Footnote 12 can be used to deduce that any pairwise outcome determined by conditions 1 and 2 is close to a particular average of the supporting profiles Footnote 13 . Thus, a cyclic outcome of transitive rankings, e.g. 8, reflects a preponderance of hypothetical cyclic contributions because it is close to an average of mostly non-transitive profiles Footnote 14 . It is possible to conclude, again without making any appeal to the transitivity of voters’ rankings, that, whenever a procedure satisfying conditions 1 and 2 produces a cycle, it

essentially ignores the specified profile with transitive preferences; instead, it reports the averages of all supporting profiles where most are cyclic (Li and Saari 2008 , p. 404).

Phases (a) to (c), in the form just described, show that being in possession of a formal approach is not the same thing as being able to control the interrelated problems that may emerge from it. The formal context that yields Sen’s result raises at the same time problems that only a focussed analysis of the constraints set by conditions 1 and 2 on the combinatorial space of pairwise rankings can solve. The analysis is not supported by the formal elements of Sen’s formulation but requires a new environment and a new associated technique. Their employment issues into a refined analysis, which is sharpened by the structural enrichment effected in phase (c).

The outcome of the geometric approach described in this section is not only a better grasp of the initial formal setting (the main goal of problem-solving in this context) but also an expansion of the types of theoretical engagements possible with Sen’s result. For instance, supporting profiles for a given outcome can now be systematically listed, transitive profiles that support cyclic outcomes can be constructed by a uniform method and higher-dimensional variations of Sen’s original example can be searched for and exhibited.

In general, phases (a) to (c) do not multiply the engagements with a given problem only in a single direction (all attention so far has been paid to cyclic outcomes) but in multiple directions. They also lead to multiple refinements of initially given problems. The next section is devoted to examining these phenomena in the context of voting problems different from those encountered so far.

4 Geometry of voting

Condorcet’s work shows that voting problems arise relatively easily from direct experimentation with hypothetical scenarios. The following one, adapted from Saari ( 2008 ), p. 2, highlights issues that require a more extensive mobilisation of structural resources than the problems considered in the preceding sections. Suppose that a company’s executive committee, composed of fifteen members, must choose one of the following investment policies: (A) invest all reserves into financial derivatives; (B) invest half of the reserves into financial derivatives; (C) invest none of the reserves into financial derivatives. Once preferences are declared, let the result be:

6 committee members prefer A to B to C;

5 committee members prefer C to B to A;

4 committee members prefer B to C to A.

Since there is no unique way of aggregating preferences, a voting procedure must be chosen for an outcome to emerge. A widely used procedure, known as plurality , counts top-ranked alternatives. Under plurality, the outcome is:

and the company’s reserves are invested into financial products. Note that, despite this outcome, most committee members consider A to be the least desirable alternative. A different voting procedure could prevent the selection of A . If, for instance, pairwise majority was adopted, the outcome would be:

i.e. the reversal of the plurality outcome. Reflection on this example (and practice with many others) leads to several questions. It is, for instance, reasonable to ask whether, by a judicious choice of procedure, it might not be possible to obtain any prescribed outcome from the same preference profile. The problem implicitly posed consists in determining how many different outcomes can be supported by the same profile, as the voting procedure varies.

A converse problem arises when a procedure is fixed and the profiles vary. The problem is now to determine the degree of variation across outcomes allowed by a fixed procedure, when all possible preference profiles are taken into account. A related problem is whether, given two distinct decision procedures, it is possible to determine outcomes, if any, that one realises and the other does not.

Finally, because the committee scenario highlights a lack of coordination between plurality and pairwise majority, the problem arises of determining whether such lack of coordination is accidental or systematic and whether it vanishes or persists in presence of other procedures (e.g. if plurality is replaced by a finer-grained rule taking second-best options into consideration).

The problems just listed are suggested by a revealing example: they do not, in themselves, deliver an explicit indication of what techniques might provide solutions. An effective response to this state of affairs consists in activities poignantly characterised by the following quote from Pólya’s work on mathematical problem-solving. The relevant activities are:

Advancing mobilization and organization of our knowledge, evolution of our conception of the problem, increasing prevision of the steps which will constitute the final argument (Pólya 1990 , p.159).

It is easy to regard the above quote as a concise characterisation of phase (a) and phase (b). To mobilise and organise knowledge is to take up the terms of the given problems and regard them symbolically in such a way that they can be integrated into a formal working environment. The environment itself displays an evolved conception of the problems, for three reasons: (i) because the originally given problems are structured within the environment; (ii) because structuring is not portraying abstractly but preparing for use or repurposing for novel theoretical and practical engagements; (iii) because the original problems are refined and new, related problems can be posed and tackled. Finally, the techniques established in phase (b), insofar as they crystallise problem-solving procedures along specific lines, make problem-solving action consequential and, insofar, increase prevision of steps in the sense of Pólya. Phase (c) then appears as a further evolution of a problem’s conception or as an evolved conception of distinct problems, which it brings together and subjects to an expanded range of techniques.

These last observations are concretely spelled out in the remainder of the present section, by looking at the way in which the voting problems just posed may be framed and attacked within a geometric environment.

4.1 Phase (a)

The problems stated at the beginning of this section concern the action of a voting procedure on profiles under a specified voting rule or a family of voting rules. It is therefore helpful to think of a procedure as a function that links profiles to outcomes under a voting rule. This observation does not provide a formal working environment but identifies its constituents, namely profiles, voting rules, outcomes and procedures. It is plausible to try and construct a formal working environment by linking them together once they have been individually structured: procedures can provide the desired link.

The first step of phase (a) consists in mobilising knowledge (in Pólya’s sense) about the items to be integrated into the environment, in order to find a useful way of presenting them as structured objects. Fixing attention on three alternatives Footnote 15 , a basic but useful remark is that they determine six possible strict rankings. Any voter can strictly rank the alternatives in only one of six ways. This is to say that information about the voters’ contribution to an election can be summarised by listing the proportions of voters expressing each of the six possible strict rankings.

If the relevant proportions are \(p_{1}, \ldots , p_{6}\) , any profile is codified by a linear combination of the form:

where \(p_{1} + \cdots + p_{6} = 1\) and \(0 \le p_{i} \le 1\) for \(i = 1, \ldots , 6\) Footnote 16 . This is enough to structure the space of profiles as a convex figure in the Euclidean space \(\mathbb {R}^{6}\) . This figure is generated by suitable combinations of six vertices. In technical terms, it is the convex hull C (6) of the six unanimity profiles \((1, 0, 0, 0, 0, 0), \ldots , (0, 0, 0, 0, 0, 1)\) .

Essentially the same line of thinking codifies the space of outcomes as C (3), the convex hull of the three unanimity outcomes (1, 0, 0), (0, 1, 0), (0, 0, 1). More concretely, C (3) is a triangular surface in \(\mathbb {R}^{3}\) , which is represented as the triangle ABC in Fig.   2 below. It is important to note that ABC is partitioned into thirteen equivalence classes, each describing the distinct realisations of the same ranking outcome, with ties included Footnote 17 . In particular, each of the six interior regions marked by the numerals 1 to 6 in Fig.   2 contains the outcomes determining one of the six possible strict rankings of A ,  B ,  C .

The space C (3) partitioned into ranking regions

An election procedure may now be regarded as a function that makes use of a voting rule to map C (6) into C (3). Not all voting rules may be codified by a convex figure but, fortunately, an entire family of them can.

By definition, a positional rule for three alternatives assigns them scores \(s_{1} \ge s_{2} \ge s_{3}\) in a non-constant fashion. Its outcome (a transitive ranking) is unaffected if a positive, linear transformation is applied to the scores. By this remark, every positional rule is identifiable with a normalised equivalent assigning scores \(1-s, s, 0\) to the alternatives, where \(s \in [0, 1/2]\) Footnote 18 . Positional rules are thus codified by vectors of the form \((1-s, s, 0)\) . When \(s = 0\) , plurality rule arises. When \(s = 1/2\) , a voting rule known as antiplurality does. In other words, by a useful quotient, positional rules can be reduced to a segment in \(\mathbb {R}^{3}\) , namely the convex hull C (2) of plurality and antiplurality.

It is worth noting that the convex figures introduced so far are the constituents of a formal working environment for positional elections, even if they are not structures in the model-theoretic sense Footnote 19 . Even though C (6),  C (3),  C (2) could be immersed in the same ambient space, they function as a working environment on their own. In fact, it is an advantage to consider them in isolation from ambient space, for the sake of working with lower-dimensional objects.

Phase (a) is completed by letting voting procedures link together C (2),  C (6) and C (3). The required link is established by the given geometrical data. To see why, let \(\mathbf{w} = (1-s, s, 0)\) be a fixed positional vector and \(\mathbf{p} = (p_{1}, \ldots , p_{6}) \in C(6)\) be a fixed profile. Each \(p_{i}\) ranks the same three candidates in one of six ways: thus \(p_{i}\) times a suitable permutation of the entries in \((1-s, s, 0)\) yields the contribution of voters of type i to a positional outcome. The outcome itself is just the sum of six contributions. Calling \(\mathbf{w} ^{1}, \ldots , \mathbf{w} ^{6}\) the permutations of \(\mathbf{w} \) ’s entries corresponding to \(p_{1}, \ldots , p_{6}\) , a plurality procedure based on the rule \(\mathbf{w} \) and acting on profile \(\mathbf{p} \) can be explicitly codified as the following function:

The link between the distinct components of the formal working environment supplied by f plays a central technical role. Because f is linear in both \(\mathbf{w} \) and \(\mathbf{p} \) , it sends convex figures like C (2) and C (6) into convex figures included in C (3). Since, moreover, the convex figures of interest are finitely generated, they can be reconstructed as convex hulls of finitely many f -values. The binding role played by f within the formal working environment opens the way to phase (b), which the next subsection examines in detail.

4.2 Phase (b)

Several related problem-solving techniques arise from the fact that f -images of convex figures are convex.

One of them, which is worthy of close examination, enables the systematic study of positional outcomes on a fixed profile. As noted earlier, the vectors \(\mathbf{w} _{0} = (1, 0, 0)\) and \(\mathbf{w} _{1/2} = (1/2, 1/2, 0)\) generate the segment C (2). For a fixed profile \(\mathbf{p} \) , the image of C (2) under f is the convex hull of the plurality outcome \(f(\mathbf{w} _{0}, \mathbf{p} )\) and the antiplurality outcome \(f(\mathbf{w} _{1/2}, \mathbf{p} )\) . More concretely, the f -image of C (2) is the segment on the triangle ABC that joins these two outcomes. In other words, the f -image of C (2) is the set of all positional outcomes for a fixed profile \(\mathbf{p} \) and can be used to study the variation of election outcomes under a change of positional procedure.

More precisely, the study of plurality outcomes rests on both the analytic (coordinate-dependent) and synthetic (coordinate-free) features of the environment. To see this, consider one of the problems raised at the beginning of the section, namely the problem of determining whether an election outcome could be changed at will by suitable changes of procedure. In general, the answer is negative: profiles that allow exactly one positional outcome can be computed. For instance, given the outcome \(A> B > C\) in region 1 from Fig.   2 , the profile (1/2, 0, 1/8, 3/8, 0, 0) can only have that outcome under every positional rule. The geometric fact guaranteeing the latter conclusion is that the f -values of plurality and antiplurality are incident on region 1: since the region is convex, the segment joining the two f -values must be included in it (similar considerations hold for any other prescribed outcome).

It is important to realise that the technique enabled by the linearity of f in the argument \(\mathbf{w} \) does not simply solve an originally given problems, but also refines it and relates to it new and subtler problems.

An immediate refinement of the given problem consists in determining the largest number of distinct plurality outcomes that any profile will allow. Thirteen distinct outcomes are possible on ABC, but convexity considerations suggest that not all of them can be realised by any one profile, since the positional outcomes must all be collinear.

The interaction between the linearity of f and the betweenness geometry of ABC provides more specific information. Let \(\ell \) be the segment joining the plurality and antiplurality outcome. If its endpoints lie on labeled, interior regions of C (3) (see Fig.   2 ), these regions can be the same or distinct. If distinct, they are either adjacent or separated by one or two regions (either counting clockwise or counterclockwise). The last case is the less straightforward one. If, in this case, \(\ell \) intersects \(\mathcal {I}\) (see fn.17), exactly three outcomes occur. Otherwise, there is a line parallel to \(\ell \) and intersecting \(\mathcal {I}\) .

The line \(\ell \) is wholly contained in one of the two half-planes determined by that parallel line (this step of the argument relies upon Pasch’s axiom). Since any such half-plane, being convex, cannot contain more than four interior regions, the maximum number of strict positional outcomes determined by a fixed p is 4. Similar considerations show that the maximum number of positional outcomes with at least one tie is 3. Thus, at most 7 different ranking outcomes can arise from the variation of a positional rule on any profile (for further details, see Saari 1994 , pp. 116–118).

A new problem connected to the original one consists in determining whether, for any given positional rules \(\mathbf{w} _{1}, \mathbf{w} _{2}\) and two ranking regions \(r_{1}, r_{2}\) on ABC, there is a profile \(\mathbf{p} \) such that \(f(\mathbf{w} _{1}, \mathbf{p} )\) lies on \(r_{1}\) and \(f(\mathbf{w} _{2}, \mathbf{p} )\) lies on \(r_{2}\) . The coarser problem concerning whether or not positional outcomes can in general be changed by a change of rule becomes here the finer problem of determining whether there are profiles especially vulnerable to manipulation.

The affirmative solution to the problem (see Saari 1994 . p. 74) is achieved by taking advantage of analytic considerations. It suffices to note that, in view of equation (1), \(f(\mathbf{w} _{1}, \mathbf{p} )\) and \(f(\mathbf{w} _{2}, \mathbf{p} )\) determine three linear equations each Footnote 20 in the six unknowns \(p_{1}, \ldots , p_{6}\) .

Because the three components of an outcome sum up to 1, when two are known the third is uniquely determined. By this observation, \(f(\mathbf{w} _{1}, \mathbf{p} )\) , \(f(\mathbf{w} _{2}, \mathbf{p} )\) are unambiguously identified by four, instead of six, linear equations. Once the linear equation \(p_{1} + \cdots + p_{6} = 1\) , which requires that the unknowns determine a profile, is added, the problem of finding profiles vulnerable to manipulation can be restated as the problem of solving a linear system in five equations and six unknowns.

Dealing with systems of linear equations makes it possible to ascertain the robustness of vulnerability to manipulation, i.e. to determine whether there are entire linear spaces of vulnerable profiles Footnote 21 or, instead, small profile variations suffice to disrupt manipulation (both cases occur, depending on whether or not outcomes involve ties).

This simple instance of problem-solving supported by the convex environments for positional procedures is touched upon to draw attention to the fact that, true to Pólya’s quote at the beginning of this section, the conception of the original voting problems has evolved. The increase in problem-solving resources does not coincide merely with an increase in solutions to pre-existing problems, but also with an increase in the store of relevant problems and the detail of their formulation.

Before considering the further expansion of problems and techniques that accompanies phase (c), it is worth noting that the environment supplied in phase (a) sustains multiple techniques. We have seen one revolving around the fact that the f -image of C (2) is a segment lying on ABC. We can also make use of the fact that the f -image of C (6) is a polygon included in ABC. More precisely, for a fixed positional rule \(\mathbf{w} \) , the f -image of C (6) is the convex hull of the vectors:

Computation of the last six f -values for a specified \(\mathbf{w} \) shows what positional outcomes that rule realises. If \(\mathbf{w} = \mathbf{w} _{0}\) , the values in (2) reduce to (1, 0, 0), (0, 1, 0), (0, 0, 1), whose convex hull is the whole of ABC. As s increases from 0 to 1/2, the set of positional outcomes shrinks until, at antiplurality, the values in (2) reduce to (1/2, 1/2, 0), (1/2, 0, 1/2) and (0, 1/2, 1/2), i.e. the intersections between a median and a side of ABC. It follows that antiplurality realises only one fourth of the plurality outcomes. Specific computations make it possible to determine sets of profiles that are not realised under one positional rule but realised under another Footnote 22 .

In this case, as well as in the previous illustrations of phase (b), to tackle an initially stated problem is not simply to solve it by bringing an enquiry to a definitive close, but rather to find a way of generating as much information as can be related to the original problem. It can be readily shown that, in general, different voting rules are associated with different families of outcomes. The geometric environment, however, does not take this information as conclusive: it takes it as a point of departure for further investigations, concerning, e.g. the proportion by which outcomes are reduced in the transition from a positional rule to another or the definability of sets of outcomes supported by a particular rule but not by another.

Moreover, the geometric environment itself is not only the support of techniques that solve, refine and expand problems, but can itself be taken as a term of a larger problem. In the present context, this happens when the goal is to carry out a comparative analysis of positional and pairwise majority outcomes. The problem of integrating the geometric environment of positional procedures with a distinct environment for pairwise majority leads to phase (c).

4.3 Phase (c)

For the sake of clarity and simplicity, it is convenient to discuss phase (c) only relative to majority on a distinguished pair of alternatives \(\{A, B\}\) and plurality (recall that \(\mathbf{w} _{0}\) denotes the plurality rule), as opposed to arbitrary positional procedures. The methods shortly to be examined could be expanded to cover other pairwise contributions and any positional method.

A geometric environment for \(\{A, B\}\) -majority is not too difficult to set up. This procedure must act on the space of profiles C (6) and may be described as follows:

where the first term of the difference computes the proportion of voters ranking A strictly above B and the second the proportion ranking B strictly above A . It follows that \(f_{A, B}\) takes values in the segment \([-1, 1]\) (other pairs of alternatives would be treated in the same way). Note that pairwise majority could have been chosen to take pairs as values, as opposed to differences, in the same way in which e.g. plurality takes triples as values.

The advantage of using a single number as \(f_{A, B}(\mathbf{p} )\) is that a comparative plurality / majority outcome only lists four, as opposed to five, numerical components. Geometrically speaking, dimension is not increased beyond necessity. When comparative outcomes are represented as ordered 4-tuples, they may be regarded as products of a plurality outcome and a majority outcome. The resulting, comparative outcome space is then the product of the outcome spaces \([-1, 1]\) and C (3), namely the prism \(\mathbb {P} = [-1, 1]\times C(3)\) , depicted in Fig.   3 .

The prism \(\mathbb {P}\) . Numerals label the unanimity outcomes under h

The new formal working environment consists of C (6), \(\mathbf{w} _{0}\) , and two simultaneous decision procedures taking values in \(\mathbb {P}\) . It is convenient to amalgamate the procedures into a single, comparative procedure \(h: C(6) \longrightarrow \mathbb {P}\) such that:

The problem of integrating the positional environment into a comparative environment is not yet completely resolved. It is not known whether the newly instituted geometric environment sustains problem-solving techniques. A reasonable way to proceed is to see how far the methods developed in phase (b) can be pushed. Convexity considerations continue to apply, since the h -image of C (6) is the convex hull generated by the values of h at the unanimity profiles. These h -values are labelled in Fig.   3 and determine a tetrahedron Footnote 23 . The analysis of positional outcomes, which could be carried out on a simple and well-behaved outcome space (i.e. the equilateral triangle ABC), is now replaced by comparative analysis of the same type on a higher-dimensional object.

The increase in dimension produced by the comparative analysis environment is an adversity that motivates the search for a way to transfer the same analysis to a lower-dimensional object. This situation provides, in a simple context, the structural analogue of a predicament familiar from modelling with differential equations, which arises when the existence of an exact but intractable solution prompts the search for manageable approximations. The relevant contrast here is not between exact and approximate but more generally between intractable and tractable. Its significance lies in the fact that intractability requires an adjustment of mathematical methods.

In our context, the problem of finding a setup for lower-dimensional comparative analysis is tackled by the construction of a suitable system of coordinates for comparative outcomes (see Saari 1994 , p. 127). The new coordinates are obtained by normalising \(p_{i}\) relative to \(d = p_{1} + p_{2} + p_{3}\) , if \(i = 1, 2, 3\) , and relative to \(1 - d\) otherwise. What is being exploited by the normalisations is the fact that voters of type 1, 2, 3 (respectively 4, 5, 6) rank A above B ( B above A ), i.e. they are of the same \(\{A, B\}\) -majority type.

Any profile \(\mathbf{p} = (p_{1}, p_{2}, p_{3}, p_{4}, p_{5}, p_{6})\) splits into the profiles

of opposite majority types. It now suffices to set:

The point \(\alpha \) lies on the triangular face of the prism \(\mathbb {P}\) containing the edge that joins 1 and 2. In a similar manner, \(\mathbf{p} _{BA}\) determines a point \(\beta \) on the edge joining 3, 4.

Comparative outcomes can now be described in terms of the majority-type coordinates \(\alpha , \beta \) . More precisely, they are convex combinations of \(\alpha \) and \(\beta \) , of the form:

The majority-type coordinates deliver a dimensional reduction for a synthetic reason, i.e. because, by Thales theorem, it is possible to project the segment joining \(\alpha \) and \(\beta \) , as well as the outcome \(q = d\alpha + (1-d)\beta \) , on a triangular face of \(\mathbb {P}\) , without altering proportions between the distances among \(\alpha , q, \beta \) .

Comparisons between majority and plurality outcomes can thus be carried out in the lower-dimensional space C (3). The use of \(\alpha , \beta \) carries its own problem-solving techniques. Since \(\alpha , \beta \) are projected on AB , AC respectively, geometrical considerations show that the midpoint of \(\alpha , \beta \) can be placed in any prescribed ranking region of C (3). It now follows that the plurality outcome for \(\{A, B\}\) is entirely independent of the corresponding majority outcome Footnote 24 . The lack of coordination between plurality and pairwise majority detected by the committee scenario at the beginning of this section is not an accident.

Much like the positional analysis in phase (b), the comparative analysis in phase (c) can be refined and extended. It is, for instance, possible to ask how decisive the victory of A over B must be for a specified positional rule not to reverse the pairwise majority outcome; it may be asked whether or not distinct positional outcomes are also independent of each other; it may be asked whether positional procedures other than plurality are better coordinated with pairwise majority outcomes, and so on. The techniques introduced are able to answer the questions just posed. Their applicability is nothing but their ability to promote new and manifold interactions with given problems.

5 Concluding remarks

Philosophers working on scientific and mathematical practice have repeatedly stressed the central role of agents (see e.g. the survey of agent-based studies by Jessica Carter in Carter ( 2019 ) and Hasok Chang’s wide-ranging discussion of agents in Chang ( 2011 )) and, as a consequence, the philosophical relevance of the activities, purposes and aims that underlie and motivate specialised enquiries.

It seems to me that the dual notion to that of an agent is the notion of a problem. The engagements of agents point to problematic situations and problem-solving techniques. Symmetrically, problems refer to activities and resources mobilised to carry them out and to achieve desired goals. Thus, one may look at agents pursuing specific aims, finding obstacles in their way and seeking to overcome them or, dually, at problems arising in the course of an enquiry, techniques deployed to tackle them, developments and revisions of such techniques.

In a similar way, phases (a) to (c), which were singled out as salient steps in problem-solving, may be regarded as activities that promote novel transactions with familiar circumstances. For instance, the problem of book classification in Sect.  2 leads to a reorganised setting in which an error-detecting code is in use. The problem of understanding the impact of decisive agents and unanimity on decision-making leads, in Sect.  3 , to designing procedures unaffected by cyclic outcomes (see e.g. Li and Saari 2008 , pp. 406–408 and (Petron and Saari 2006 , pp. 274–279). Finally, the geometric techniques explored in Sect.  4 lead to new analyses of political elections (an application to presidential elections in the USA may be found in Tabarrok ( 2001 )) and new evaluations of decision procedures.

The expansion of activities is concomitant with the organisation of knowledge. Subject matter that initially appears under the guise of brute facts or puzzling circumstances acquires structural properties as a result of an agent-driven endeavour. The fact, stressed by Dewey, that ‘new formal properties accrue to subject matter in virtue of its subjection to certain types of operation’ (Dewey 2008 , p. 105) is a distinctive feature of scientific enquiry that highlights its formative character.

Again, two complementary perspectives can be taken, depending on whether one wishes to stress the formative effects of an agent’s intervention on certain materials or the formative functions associated with the development of problems through phases (a) to (c). In both cases, the materials of experience are structured in the course of a continuous process; at the same time, their newly acquired structural organisation refines and regulates the further conduct of enquiry.

A small sample of important contributions is Ardourel and Jebeile ( 2017 ), Batterman ( 2006 ), Bursten ( 2021 ), Fillion ( 2019 ), Lenhard ( 2007 ), Wilson ( 2006 ).

The single, most extensive discussion of mathematical modelling in social science is the study of econometric and macroeconomic models in Boumans ( 2005 ). Here models are typically systems of equations, rather than the structured objects discussed in this paper.

It may be noted that the views expressed in these contributions revive, in effect, some elements of an account of mathematical modelling offered in Coombs et al. ( 1954 ).

Although, in the absence of any limits to the mathematical description of an empirical setup, applications of mathematics can proceed without the immersion and interpretation steps prescribed by the inferential model advocated in Bueno and Colyvan ( 2011 ).

Error-detection dispenses with checks against an authentic copy of the codeword.

More explicitly, with addition and multiplication modulo 11. Sums and products are computed by taking the remainders of division by 11 of the usual arithmetical sums and products. Multiplication modulo 10 rules out field structure because it allows zero divisors, e.g. \(2\cdot 5 = 0\) modulo 10.

A code is just a set of codewords. Obviously, useful codes have distinctive features.

If the value of \(x_{10}\) required by the condition below is 10, the symbol X is used.

It may be worth noting that was just described is the familiar ISBN code.

See Li and Saari ( 2008 ), pp.409-411.

In fact, the literature on Sen’s theorem predating Li and Saari ( 2008 ) has often had to struggle with fairly intricate constructions of cycles from transitive information (see e.g. Gibbard ( 1974 ) and Gaertner et al. ( 1991 )).

For a more formal definition of ‘closeness’ , see the Appendix.

See the Appendix or Li and Saari ( 2008 ), p.411 for details.

With two voters, three out of the four supporting profiles (8, 8), (2, 6), (2, 8), (8, 6) contain cyclic rankings.

The methods described below can be adapted to any finite number of alternatives, but they are especially transparent when restricted to three.

In the remainder of the paper, the following assignment of indices to voters’ rankings is adopted: 1 labels \(A> B > C\) ; 2 labels \(A> C > B\) ; 3 labels \(C> A > B\) ; 4 labels \(B> A > C\) ; 5 labels \(B> C > A\) ; 6 labels \(C> B > A\) .

To see this, consider the midpoint (1/2, 1/2, 0) of AB , marked in Fig.   2 : clearly, it identifies the ranking \(A \sim B \succ C\) , where A ,  B are tied and C receives no votes. The points on AB to the left of the midpoint, with the exception of A , assign a greater weight to A than they do to B and zero weight only to C , so they all express \(A \succ B \succ C\) . The same ranking is expressed by every point in the interior region marked by the label 1 in Fig.   2 . An entirely symmetrical situation (interchanging the roles of A ,  B ) arises to the right of the AB -midpoint and in the interior region 6. The other labeled regions can be treated in a similar manner (for instance, region 2 expresses the ranking outcome \(A \succ C \succ B\) ). The barycentre \(\mathcal {I} = (1/3, 1/3, 1/3)\) of ABC (the intersection of its medians) expresses the complete tie and the points along each median express exactly one tie between two alternatives. The six interior regions (with the respective open boundaries), the six semi-open segments (i.e. minus the endpoint \(\mathcal {I}\) ) in which the medians cut each other and the point \(\mathcal {I}\) are the 13 ranking regions into which C (3) is partitioned.

For the argument leading to this representation, see Saari ( 1994 ), pp. 47–48.

The ambient space for the convex figures that have been singled out is endowed at least with linear structure. Then, neither of C (6),  C (3),  C (2) is a substructures of the space. None of them contains the null vector or is closed under multiplication by a scalar.

Once vector components are taken separately.

Note that, when linear spaces of solutions exist, their elements with rational components are legitimate profiles.

For instance, if \(1/2 < a \le 1\) and \(b = c\) , then ( a ,  b ,  c ) is not an anti-plurality outcome but it certainly is a plurality outcome.

They are: \(1= ( (1, 0, 0), 1);\,\, 2 = (( 0, 0, 1), 1); \,\, 3 = ((0, 0, 1), -1);\,\, 4 = ((0, 1, 0), -1)\) .

An election in which a strong majority prefers A over B may see A lose to B under the plurality rule.

Ardourel, V., Barberousse, A., Imbert, C.: Inferential power, formalisms, and scientific models (2018). http://philsci-archive.pitt.edu/15165/

Ardourel, V., & Jebeile, J. (2017). On the presumed superiority of analytical solutions over numerical methods. European Journal for Philosophy of Science, 7 , 201–220.

Article   Google Scholar  

Arrow, K. (1951). Social choice and individual values . Hoboken: Wiley.

Google Scholar  

Batterman, R. (2006). On the specialness of special functions (the nonrandom effusions of the divine mathematician). The British Journal for the Philosophy of Science, 58 , 263–286.

Boumans, M. (2005). How economicsts model the world into numbers . London: Routledge.

Bueno, O., & Colyvan, M. (2011). An inferential conception of the application of mathematics. Noûs, 45 , 345–374.

Bursten, J. (2021). The function of boundary conditions in the physical sciences. Philosophy of Science, 88 , 234–257.

Carter, J. (2019). Philosophy of mathematical practice - motivations, themes and prospects. Philosophia Mathematica, 27 , 1–32.

Chang, H. (2011). The philosophical grammar of scientific practice. International Studies in the Philosophy of Science, 25 , 205–221.

Condorcet, J.A.N.d.C.m.d.: Essai sur l’application de l’analyse à la probabilité des décisions rendue à la pluralité des voix. L’Imprimerie Royale (1785)

Coombs, C., Raiffa, H., & Thrall, R. (1954). Some views on mathematical models and measurement theory. Psychological Review, 61 , 132–144.

Dewey, J. (2008). The Later Works of John Dewey, 1925–1953: 1938 – Logic: the Theory of Inquiry, edited by J.A. Boydston. Southern Illinois Press.

Field, H. (1980). Science without numbers . Oxford: Blackwell.

Fillion, N. (2019). Conceptual and computational mathematics. Philosophia Mathematica, 27 , 199–218.

Fillion, N., & Corless, R. M. (2014). On the epistemological analysis of modeling and computational error in the mathematical sciences. Synthese, 191 , 537–546.

Gaertner, W., Pattanaik, P., & Suzumura, K. (1991). Individual rights revisited. Economica, 59 , 161–177.

Gibbard, A. (1974). A pareto-consistent libertarian claim. Journal of Economic Theory, 7 , 388–410.

Lange, M. (2013). What makes a scientific explanation distinctively mathematical? The British Journal for the Philosophy of Science, 64 , 485–511.

Lange, M. (2017). Because without cause: non-causal explanations in science and mathematics . Oxford: Oxford University Press.

Lenhard, J. (2007). Computer simulation: The cooperation between experimenting and modeling. Philosophy of Science, 74 , 176–194.

Li, L. I., & Saari, D. (2008). Sen’s theorem: Geometric proof, new interpretations. Social Choice and Welfare, 31 , 393–413.

Lyon, A. (2012). Mathematical explanation of empirical facts, and mathematical realism. Australasian Journal of Philosophy, 90 , 559–578.

Petron, A., & Saari, D. G. (2006). Negative externalities and sen’s liberalism theorem. Economic Theory, 28 , 265–281.

Pincock, C. (2012). Mathematics and scientific representation . Oxford: Oxford University Press.

Book   Google Scholar  

Pólya, G. (1990). How to solve it (2nd ed.). New York: Penguin.

Saari, D. (1994). Geometry of Voting . New York: Springer.

Saari, D. (2008). Disposing dictators, demystifying voting paradoxes . Cambridge: Cambridge University Press.

Sen, A. (1970). The impossibility of a paretian liberal. Journal of Political Economy, 78 , 152–157.

Tabarrok, A. (2001). President Perot or fundamentals of voting theory illustrated with the 1992 election. Public Choice, 106 , 275–297.

Wilson, M. (2006). Wandering Significance . Oxford: Oxford University Press.

Download references

Author information

Authors and affiliations.

School of Politics, Philosophy, Language and Communication Studies, University of East Anglia, Norwich, UK

Davide Rizza

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Davide Rizza .

Ethics declarations

Human or animal rights.

I confirm that the paper ‘Applications in practice’ has not been supported by any sources of funding other than the paid research time included in my current academic contract and that, to the best of my knowledge and awareness, no conflicts of interest affect it. I also confirm that the research contained in the paper I have submitted for consideration did not involve any human participants or animals.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Given two voters and three alternatives, any pairwise outcome determined by conditions 1 and 2 is the average of all averaged supporting profiles.

We focus on the top face F of the unit cube from Fig.  1 , selected by the unanimous \(C > A\) ranking. The decisive voters impose the ( A ,  B ) and ( B ,  C ) rankings respectively. Think of F as a region of \(\mathbb {R}^{3}\) . The midpoints of F ’s sides represent ties between alternatives (e.g. the midpoint of the edge joining 2 and 8 represents \(A \sim B\) ). Take the basic open sets in F to be open rectangles. If \(a \in F\) and x is a vertex of F , a is close to x if, and only if, there is a basic open set U such that its closure contains a ,  x and no other vertex than x . Note that a can be close to more than a single vertex. If a ,  x are as above, a expresses x if, and only if, a is close only to x . Points close to exactly one vertex express its ranking. The cycle 7 at vertex (1, 1, 1), is supported by four distinct profiles. The voter decisive on A ,  B supports 7 either by the ranking 7 or the ranking 3, whose coordinates are (1, 1, 1) and (1, 0, 1) respectively. The voter decisive on B ,  C supports 7 either by the ranking 7 or the ranking 5, whose coordinates are (1, 1, 1), (0, 1, 1). If we average each of the four supporting profiles (i.e. take the average of its two components), each resulting average expresses 7. The average of these points is:

which expresses 7. The argument is similar for vertices other than 7.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Rizza, D. Mathematical problem-solving in scientific practice. Synthese 199 , 13621–13641 (2021). https://doi.org/10.1007/s11229-021-03392-1

Download citation

Received : 09 March 2021

Accepted : 28 August 2021

Published : 23 September 2021

Issue Date : December 2021

DOI : https://doi.org/10.1007/s11229-021-03392-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mathematical practice
• Scientific practice
• Applications
• Voting theory
• Find a journal
• Publish with us
• Track your research

Multiple goals, multiple solutions, plenty of second-guessing and revising − here’s how science really works

Professor of Philosophy, University of Montana

Disclosure statement

Soazig Le Bihan receives funding from the Maureen and Mike Mansfield Center at the University of Montana.

University of Montana provides funding as a member of The Conversation US.

View all partners

A man in a lab coat bends under a dim light, his strained eyes riveted onto a microscope. He’s powered only by caffeine and anticipation.

This solitary scientist will stay on task until he unveils the truth about the cause of the dangerous disease quickly spreading through his vulnerable city. Time is short, the stakes are high, and only he can save everyone. …

That kind of romanticized picture of science was standard for a long time. But it’s as far from actual scientific practice as a movie’s choreographed martial arts battle is from a real fistfight.

For most of the 20th century, philosophers of science like me maintained somewhat idealistic claims about what good science looks like. Over the past few decades, however, many of us have revised our views to better mirror actual scientific practice .

An update on what to expect from actual science is overdue. I often worry that when the public holds science to unrealistic standards, any scientific claim failing to live up to them arouses suspicion. While public trust is globally strong and has been for decades, it has been eroding. In November 2023, Americans’ trust in scientists was 14 points lower than it had been just prior to the COVID-19 pandemic, with its flurry of confusing and sometimes contradictory science-related messages.

When people’s expectations are not met about how science works, they may blame scientists. But modifying our expectations might be more useful. Here are three updates I think can help people better understand how science actually works. Hopefully, a better understanding of actual scientific practice will also shore up people’s trust in the process.

The many faces of scientific research

First, science is a complex endeavor involving multiple goals and associated activities.

Some scientists search for the causes underlying some observable effect, such as a decimated pine forest or the Earth’s global surface temperature increase .

Others may investigate the what rather than the why of things. For example, ecologists build models to estimate gray wolf abundance in Montana . Spotting predators is incredibly challenging. Counting all of them is impractical. Abundance models are neither complete nor 100% accurate – they offer estimates deemed good enough to set harvesting quotas. Perfect scientific models are just not in the cards .

Beyond the what and the why, scientists may focus on the how. For instance, the lives of people living with chronic illnesses can be improved by research on strategies for managing disease – to mitigate symptoms and improve function, even if the true causes of their disorders largely elude current medicine.

It’s understandable that some patients may grow frustrated or distrustful of medical providers unable to give clear answers about what causes their ailment. But it’s important to grasp that lots of scientific research focuses on how to effectively intervene in the world to reach some specific goals.

Simplistic views represent science as solely focused on providing causal explanations for the various phenomena we observe in this world. The truth is that scientists tackle all kinds of problems, which are best solved using different strategies and approaches and only sometimes involve full-fledged explanations.

Complex problems call for complex solutions

The second aspect of scientific practice worth underscoring is that, because scientists tackle complex problems, they don’t typically offer one unique, complete and perfect answer. Instead they consider multiple, partial and possibly conflicting solutions.

Scientific modeling strategies illustrate this point well. Scientific models typically are partial, simplified and sometimes deliberately unrealistic representations of a system of interest. Models can be physical, conceptual or mathematical. The critical point is that they represent target systems in ways that are useful in particular contexts of inquiry. Interestingly, considering multiple possible models is often the best strategy to tackle complex problems.

Scientists consider multiple models of biodiversity , atomic nuclei or climate change . Returning to wolf abundance estimates, multiple models can also fit the bill. Such models rely on various types of data, including acoustic surveys of wolf howls, genetic methods that use fecal samples from wolves, wolf sightings and photographic evidence, aerial surveys, snow track surveys and more.

Weighing the pros and cons of various possible solutions to the problem of interest is part and parcel of the scientific process. Interestingly, in some cases, using multiple conflicting models allows for better predictions than trying to unify all the models into one.

The public may be surprised and possibly suspicious when scientists push forward multiple models that rely on conflicting assumptions and make different predictions. People often think “real science” should provide definite, complete and foolproof answers to their questions. But given various limitations and the world’s complexity, keeping multiple perspectives in play is most often the best way for scientists to reach their goals and solve the problems at hand.

Science as a collective, contrarian endeavor

Finally, science is a collective endeavor, where healthy disagreement is a feature, not a bug.

The romanticized version of science pictures scientists working in isolation and establishing absolute truths. Instead, science is a social and contrarian process in which the community’s scrutiny ensures we have the best available knowledge. “Best available” does not mean “definitive,” but the best we have until we find out how to improve it. Science almost always allows for disagreements among experts.

Controversies are core to how science works at its best and are as old as Western science itself. In the 1600s, Descartes and Leibniz fought over how to best characterize the laws of dynamics and the nature of motion.

The long history of atomism provides a valuable perspective on how science is an intricate and winding process rather than a fast-delivery system of results set in stone. As Jean Baptiste Perrin conducted his 1908 experiments that seemingly settled all discussion regarding the existence of atoms and molecules, the questions of the atom’s properties were about to become the topic of decades of controversies with the birth of quantum physics.

The nature and structure of fundamental particles and associated fields have been the subject of scientific research for more than a century. Lively academic discussions abound concerning the difficult interpretation of quantum mechanics , the challenging unification of quantum physics and relativity , and the existence of the Higgs boson , among others.

Distrusting researchers for having healthy scientific disagreements is largely misguided.

A very human practice

To be clear, science is dysfunctional in some respects and contexts. Current institutions have incentives for counterproductive practices, including maximizing publication numbers . Like any human endeavor, science includes people with bad intent, including some trying to discredit legitimate scientific research . Finally, science is sometimes inappropriately influenced by various values in problematic ways.

These are all important considerations when evaluating the trustworthiness of particular scientific claims and recommendations. However, it is unfair, sometimes dangerous, to mistrust science for doing what it does at its best. Science is a multifaceted endeavor focused on solving complex problems that typically just don’t have simple solutions. Communities of experts scrutinize those solutions in hopes of providing the best available approach to tackling the problems of interest.

Science is also a fallible and collective process. Ignoring the realities of that process and holding science up to unrealistic standards may result in the public calling science out and losing trust in its reliability for the wrong reasons.

• Philosophy of science
• Scientific research
• Science myths
• Trust in science

Casual Facilitator: GERRIC Student Programs - Arts, Design and Architecture

Senior Lecturer, Digital Advertising

Associate Professor, Psychology

Service Delivery Fleet Coordinator

Manager, Centre Policy and Translation

How to Solve It

30% off with code PUP30

Before you purchase audiobooks and ebooks

Please note that audiobooks and ebooks purchased from this site must be accessed on the Princeton University Press app. After you make your purchase, you will receive an email with instructions on how to download the app. Learn more about audio and ebooks .

Support your local independent bookstore.

• United States
• United Kingdom

Mathematics

How to Solve It: A New Aspect of Mathematical Method

• John H. Conway

The bestselling book that has helped millions of readers solve any problem

• Princeton Science Library

• Request Exam Copy
• Download Cover

A must-have guide by eminent mathematician G. Polya, How to Solve It shows anyone in any field how to think straight. In lucid and appealing prose, Polya reveals how the mathematical method of demonstrating a proof or finding an unknown can help you attack any problem that can be reasoned out—from building a bridge to winning a game of anagrams. How to Solve It includes a heuristic dictionary with dozens of entries on how to make problems more manageable—from analogy and induction to the heuristic method of starting with a goal and working backward to something you already know. This disarmingly elementary book explains how to harness curiosity in the classroom, bring the inventive faculties of students into play, and experience the triumph of discovery. But it’s not just for the classroom. Generations of readers from all walks of life have relished Polya’s brilliantly deft instructions on stripping away irrelevancies and going straight to the heart of a problem.

"Every prospective teacher should read it. In particular, graduate students will find it invaluable. The traditional mathematics professor who reads a paper before one of the Mathematical Societies might also learn something from the book: 'He writes a, he says b, he means c; but it should be d.'"—E. T. Bell, Mathematical Monthly

"[ How to Solve It ] shows anew how keen its author is on questions of method and the formulation of methodological principles. Exposition and illustrative material are of a disarmingly elementary character, but very carefully thought out and selected."—Herman Weyl, Mathematical Review

"I recommend it highly to any person who is seriously interested in finding out methods of solving problems, and who does not object to being entertained while he does it."— Scientific Monthly

"Every mathematics student should experience and live this book"— Mathematics Magazine

"In an age that all solutions should be provided with the least possible effort, this book brings a very important message: mathematics and problem solving in general needs a lot of practice and experience obtained by challenging creative thinking, and certainly not by copying predefined recipes provided by others. Let's hope this classic will remain a source of inspiration for several generations to come."—A. Bultheel, European Mathematical Society

“Every mathematics student should experience and live this book.”— Mathematics Magazine

“[ How to Solve It ] shows anew how keen its author is on questions of method and the formulation of methodological principles. Exposition and illustrative material are of a disarmingly elementary character, but very carefully thought out and selected.”—Herman Weyl, Mathematical Review

“Any young person seeking a career in the sciences would do well to ponder this important contribution to the teacher’s art.”—A. C. Schaeffer, American Journal of Psychology

“I recommend it highly to any person who is seriously interested in finding out methods of solving problems, and who does not object to being entertained while he does it.”— Scientific Monthly

“Every prospective teacher should read [ How to Solve It ]. In particular, graduate students will find it invaluable. The traditional mathematics professor who reads a paper before one of the Mathematical Societies might also learn something from the book: ‘He writes a, he says b, he means c; but it should be d.’”—E. T. Bell, Mathematical Monthly

“In an age that all solutions should be provided with the least possible effort, this book brings a very important message: mathematics and problem solving in general needs a lot of practice and experience obtained by challenging creative thinking, and certainly not by copying predefined recipes provided by others. Let’s hope this classic will remain a source of inspiration for several generations to come.”—A. Bultheel, European Mathematical Society

Stay connected for new books and special offers. Subscribe to receive a welcome discount for your next order. 

• ebook & Audiobook Cart

E-Paper | August 16, 2024

Solving problems

• DOI: 10.1007/s11075-024-01905-w
• Corpus ID: 271624586

Variable parameter Uzawa method for solving the indefinite least squares problem

• Lingsheng Meng , Kailiang Xin , Jun Li
• Published in Numerical Algorithms 31 July 2024
• Mathematics, Computer Science

12 References

Algorithms for indefinite linear least squares problems, ussor method for solving the indefinite least squares problem, backward error and condition number analysis for the indefinite linear least squares problem, on the partial condition numbers for the indefinite least squares problem, block sor methods for the solution of indefinite least squares problems, incomplete hyperbolic gram-schmidt-based preconditioners for the solution of large indefinite least squares problems, preconditioned conjugate gradient methods for the solution of indefinite least squares problems, a backward stable hyperbolic qr factorization method for solving indefinite least squares problem, two new variants of nonlinear inexact uzawa algorithms for saddle-point problems, inertia properties of indefinite quadratic forms, related papers.

Showing 1 through 3 of 0 Related Papers

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Solving the vehicle routing problem with stochastic travel cost using deep reinforcement learning.

1. Introduction

2. preliminary, 2.1. vrp-stc, 2.2. reinforcement learning framework, 3.1. formulation of drl.

• State: Status s t = ( R t , C t ) is a part of the solution, for instance, G ( Q , q , R ) created at time step t . Here, R t (for t ≠ 0) is the group of customers receiving services, which includes all of the chosen customer locations up to step t , C t is the set of candidate nodes at step t , Q is the maximum capacity of each vehicle, q is the demand of customers.
• Action: Action a t indicates that at step t , the candidate node π t is chosen from the candidate node set C t and added to the service customer set R t .
• Transition: With the action a t , a modern fractional solution is obtained as the following state, i.e., s t+ 1 = ( R t+ 1 , C t+ 1 ). Within the updated state, R t+ 1 includes π t in addition to the nodes chosen so far, whereas C t+ 1 consists of the candidate nodes from C t with π t expelled.
• Reward: To minimize the total cost, we define the value of the objective function at step t as Obj t = min E( cost ), and the reward at step t as r t = Obj t−1 − Obj t .
• Policy: The strategy P θ is parameterized using θ within the GAT-AM model. At each step t , a candidate node is automatically chosen as the service customer node until all service customer nodes are chosen, resulting in the final solution π = { π 1 , π 2 , …, π n ,} generated by the policy.

3.3. Encoder

3.4. decoder, 3.5. algorithm.

3.6. Stochastic Travel Costs

4. experiments, 4.1. experimental settings, 4.2. baseline methods and evaluation metrics, 4.3. comparison analysis, 4.4. model convergence performance, 4.5. visualization, 5. conclusions.

• Algorithmic Refinement and Generalization: The model currently exhibits certain limitations. Enhancing the algorithm’s generalization capacity to accommodate a broader spectrum of environments and various problem scenarios represents a pivotal area for future investigation.
• Real-time Dynamic Planning: With the growing demand for practical applications, implementing real-time dynamic planning within the model to accommodate the evolving logistics demands and traffic conditions is a pressing challenge awaiting resolution.
• Multi-objective Optimization: The present model primarily focuses on minimizing the total travel cost. In the future, exploration could be extended to achieve a balance among multiple objectives, such as service level assurance and minimizing environmental impacts.

Author Contributions

Data availability statement, acknowledgments, conflicts of interest.

• Khalil, E.; Dai, H.; Zhang, Y.; Dilkina, B.; Song, L. Learning combinatorial optimization algorithms over graphs. Adv. Neural Inf. Process. Syst. 2017 , 30 , 6348–6358. [ Google Scholar ]
• Bengio, Y.; Lodi, A.; Prouvost, A. Machine learning for combinatorial optimization: A methodological tour d’horizon. Eur. J. Oper. Res. 2021 , 290 , 405–421. [ Google Scholar ] [ CrossRef ]
• Glorot, X.; Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy, 13–15 May 2010; pp. 249–256. [ Google Scholar ]
• Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction. IEEE Trans. Neural Netw. 1998 , 9 , 1054. [ Google Scholar ] [ CrossRef ]
• Silver, D.; Schrittwieser, J.; Simonyan, K.; Antonoglou, I.; Huang, A. Mastering the game of Go without human knowledge. Nature 2017 , 550 , 354–359. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.A.; Veness, J.; Bellemare, M.G.; Graves, A.; Riedmiller, M.; Fidjeland, A.K.; Ostrovski, G. Human-level control through deep reinforcement learning. Nature 2015 , 518 , 529–533. [ Google Scholar ] [ CrossRef ]
• Karimi-Mamaghan, M.; Mohammadi, M.; Meyer, P.; Karimi-Mamaghan, A.M.; Talbi, E.-G. Machine learning at the service of meta-heuristics for solving combinatorial optimization problems: A state-of-the-art. Eur. J. Oper. Res. 2022 , 296 , 393–422. [ Google Scholar ] [ CrossRef ]
• Watkins, C.J.; Dayan, P. Q-learning. Mach. Learn. 1992 , 8 , 279–292. [ Google Scholar ] [ CrossRef ]
• Williams, R.J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 1992 , 8 , 229–256. [ Google Scholar ] [ CrossRef ]
• Konda, V.; Tsitsiklis, J. Actor-critic algorithms. Adv. Neural Inf. Process. Syst. 1999 , 12 , 1008–1014. [ Google Scholar ]
• Silver, D.; Lever, G.; Heess, N.; Degris, T.; Wierstra, D.; Riedmiller, M. Deterministic policy gradient algorithms. In Proceedings of the International Conference on Machine Learning, Beijing, China, 22–24 June 2014; pp. 387–395. [ Google Scholar ]
• Schulman, J.; Levine, S.; Abbeel, P.; Jordan, M.; Moritz, P. Trust region policy optimization. In Proceedings of the International Conference on Machine Learning, Lille, France, 7–9 July 2015; pp. 1889–1897. [ Google Scholar ]
• Lillicrap, T.P.; Hunt, J.J.; Pritzel, A.; Heess, N.; Erez, T.; Tassa, Y.; Silver, D.; Wierstra, D. Continuous control with deep reinforcement learning. arXiv 2015 , arXiv:1509.02971. [ Google Scholar ]
• Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; Klimov, O. Proximal policy optimization algorithms. arXiv 2017 , arXiv:1707.06347. [ Google Scholar ]
• Babaeizadeh, M.; Frosio, I.; Tyree, S.; Clemons, J.; Kautz, J. Reinforcement learning through asynchronous advantage actor-critic on a gpu. arXiv 2016 , arXiv:1611.06256. [ Google Scholar ]
• Levine, S.; Finn, C.; Darrell, T.; Abbeel, P. End-to-end training of deep visuomotor policies. J. Mach. Learn. Res. 2016 , 17 , 1334–1373. [ Google Scholar ]
• Deng, Y.; Bao, F.; Kong, Y.; Ren, Z.; Dai, Q. Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. Syst. 2016 , 28 , 653–664. [ Google Scholar ] [ CrossRef ]
• Zheng, G.; Zhang, F.; Zheng, Z.; Xiang, Y.; Yuan, N.J.; Xie, X.; Li, Z. DRN: A deep reinforcement learning framework for news recommendation. In Proceedings of the 2018 World Wide Web Conference, Lyon, France, 23–27 April 2018; pp. 167–176. [ Google Scholar ]
• Silver, D.; Huang, A.; Maddison, C.J.; Guez, A.; Sifre, L.; Van Den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M. Mastering the game of Go with deep neural networks and tree search. Nature 2016 , 529 , 484–489. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Silver, D.; Hubert, T.; Schrittwieser, J.; Antonoglou, I.; Lai, M.; Guez, A.; Lanctot, M.; Sifre, L.; Kumaran, D.; Graepel, T. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 2018 , 362 , 1140–1144. [ Google Scholar ] [ CrossRef ]
• Schrittwieser, J.; Antonoglou, I.; Hubert, T.; Simonyan, K.; Sifre, L.; Schmitt, S.; Guez, A.; Lockhart, E.; Hassabis, D.; Graepel, T. Mastering atari, go, chess and shogi by planning with a learned model. Nature 2020 , 588 , 604–609. [ Google Scholar ] [ CrossRef ]
• Vinyals, O.; Fortunato, M.; Jaitly, N. Pointer networks. Adv. Neural Inf. Process. Syst. 2015 , 28 . [ Google Scholar ] [ CrossRef ]
• Lu, H.; Zhang, X.; Yang, S. A learning-based iterative method for solving vehicle routing problems. In Proceedings of International Conference on Learning Representations. Available online: https://openreview.net/forum?id=BJe1334YDH (accessed on 13 August 2024).
• Manchanda, S.; Mittal, A.; Dhawan, A.; Medya, S.; Ranu, S.; Singh, A. Learning heuristics over large graphs via deep reinforcement learning. arXiv 2019 , arXiv:1903.03332. [ Google Scholar ]
• Mazyavkina, N.; Sviridov, S.; Ivanov, S.; Burnaev, E. Reinforcement learning for combinatorial optimization: A survey. Comput. Oper. Res. 2021 , 134 , 105400. [ Google Scholar ] [ CrossRef ]
• Cappart, Q.; Chételat, D.; Khalil, E.B.; Lodi, A.; Morris, C.; Veličković, P. Combinatorial optimization and reasoning with graph neural networks. J. Mach. Learn. Res. 2023 , 24 , 1–61. [ Google Scholar ]
• Kool, W.; Van Hoof, H.; Welling, M. Attention, learn to solve routing problems! arXiv 2018 , arXiv:1803.08475. [ Google Scholar ]
• Nowak, A.; Villar, S.; Bandeira, A.S.; Bruna, J. A note on learning algorithms for quadratic assignment with graph neural networks. Stat 2017 , 1050 , 22. [ Google Scholar ]
• Li, Z.; Chen, Q.; Koltun, V. Combinatorial optimization with graph convolutional networks and guided tree search. Adv. Neural Inf. Process. Syst. 2018 , 31 . Available online: https://proceedings.neurips.cc/paper_files/paper/2018/file/8d3bba7425e7c98c50f52ca1b52d3735-Paper.pdf (accessed on 13 August 2024).
• Drori, I.; Kharkar, A.; Sickinger, W.R.; Kates, B.; Ma, Q.; Ge, S.; Dolev, E.; Dietrich, B.; Williamson, D.P.; Udell, M. Learning to solve combinatorial optimization problems on real-world graphs in linear time. In Proceedings of the 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA), Miami, FL, USA, 14–17 December 2020; pp. 19–24. [ Google Scholar ]
• Lodi, A.; Mossina, L.; Rachelson, E. Learning to handle parameter perturbations in combinatorial optimization: An application to facility location. EURO J. Transp. Logist. 2020 , 9 , 100023. [ Google Scholar ] [ CrossRef ]
• Xidias, E.; Zacharia, P.; Nearchou, A. Intelligent fleet management of autonomous vehicles for city logistics. Appl. Intell. 2022 , 52 , 18030–18048. [ Google Scholar ] [ CrossRef ]
• Luo, H.; Dridi, M.; Grunder, O. A branch-price-and-cut algorithm for a time-dependent green vehicle routing problem with the consideration of traffic congestion. Comput. Ind. Eng. 2023 , 177 , 109093. [ Google Scholar ] [ CrossRef ]
• Bai, R.; Chen, X.; Chen, Z.-L.; Cui, T.; Gong, S.; He, W.; Jiang, X.; Jin, H.; Jin, J.; Kendall, G. Analytics and machine learning in vehicle routing research. Int. J. Prod. Res. 2023 , 61 , 4–30. [ Google Scholar ] [ CrossRef ]
• Puterman, M.L. Markov decision processes. Handb. Oper. Res. Manag. Sci. 1990 , 2 , 331–434. [ Google Scholar ]
• Veličković, P.; Cucurull, G.; Casanova, A.; Romero, A.; Lio, P.; Bengio, Y. Graph attention networks. arXiv 2017 , arXiv:1710.10903. [ Google Scholar ]
• He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [ Google Scholar ]
• Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. arXiv 2017 , arXiv:1706.03762. [ Google Scholar ]
• Rennie, S.J.; Marcheret, E.; Mroueh, Y.; Ross, J.; Goel, V. Self-critical sequence training for image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 7008–7024. [ Google Scholar ]
• Liu, Z.; Li, X.; Khojandi, A. The flying sidekick traveling salesman problem with stochastic travel time: A reinforcement learning approach. Transp. Res. Part E Logist. Transp. Rev. 2022 , 164 , 102816. [ Google Scholar ] [ CrossRef ]
• Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014 , arXiv:1412.6980. [ Google Scholar ]
• Helsgaun, K. An extension of the Lin-Kernighan-Helsgaun TSP solver for constrained traveling salesman and vehicle routing problems. Rosk. Rosk. Univ. 2017 , 12 , 966–980. [ Google Scholar ]
• Bello, I.; Pham, H.; Le, Q.V.; Norouzi, M.; Bengio, S. Neural combinatorial optimization with reinforcement learning. arXiv 2016 , arXiv:1611.09940. [ Google Scholar ]

Click here to enlarge figure

Share and Cite

Cai, H.; Xu, P.; Tang, X.; Lin, G. Solving the Vehicle Routing Problem with Stochastic Travel Cost Using Deep Reinforcement Learning. Electronics 2024 , 13 , 3242. https://doi.org/10.3390/electronics13163242

Cai H, Xu P, Tang X, Lin G. Solving the Vehicle Routing Problem with Stochastic Travel Cost Using Deep Reinforcement Learning. Electronics . 2024; 13(16):3242. https://doi.org/10.3390/electronics13163242

Cai, Hao, Peng Xu, Xifeng Tang, and Gan Lin. 2024. "Solving the Vehicle Routing Problem with Stochastic Travel Cost Using Deep Reinforcement Learning" Electronics 13, no. 16: 3242. https://doi.org/10.3390/electronics13163242

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Grab your spot at the free arXiv Accessibility Forum

Help | Advanced Search

Mathematics > Optimization and Control

Title: an efficient integer programming model for solving the master planning problem of container vessel stowage.

Abstract: A crucial role of container shipping is maximizing container uptake onto vessels, optimizing the efficiency of a fundamental part of the global supply chain. In practice, liner shipping companies include block stowage patterns that ensure that containers in above and below deck partitions of bays have the same destination. Despite preventing restows, increasing free space, and benefits for crane makespan and hydrostatics, this practical planning requirement is rarely included in stowage optimization models. In our paper, we introduce a novel 0-1 IP model that searches in the space of valid paired block stowage patterns, named template planning, which ensures sufficient vessel capacity and limits to crane makespan, trim, and bending moment. Our results show that template planning outperforms traditional allocation planning concerning optimality and runtime efficiency while preserving a sufficiently accurate representation of master planning constraints and objectives.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .