Methods of Assessment

Knowledge of the primary area and secondary readings will be demonstrated across two component assessments designed in a manner agreed upon by the student and advisory committee. These assessments may take the form of written exam questions, integrative essays, research proposals, etc. The demonstrated knowledge of the primary and secondary areas may be distributed across the two component assessments but there should be minimal overlap in the content of the two assessments.

For example, to demonstrate knowledge of the primary area, the student may prepare a broad, integrative position paper concerning the area of primary research. This could take the form of an essay or a grant proposal based on a standard format (CIHR, NSERC). The expected length of the main body of the text is 3,000 to 6,000 words. The submitted document must include a bibliography of all works cited.  

Knowledge of the secondary readings is similarly flexible. For example, take home questions could be provided for each substantive area covered by the secondary readings, or a grant proposal produced that is framed to substantively encompass the secondary readings.

An oral defense is required in connection with both components of the candidacy assessment; the student's advisory committee or designates will serve as examiners. Example models for the two components of the candidacy assessment are provided at the end of this document.

Once reading lists and methods of evaluation are determined for primary and secondary readings, an assessment document that 1) includes the reading lists, 2) provides clear instructions regarding the methods of assessment, and 3) states all assessment deadlines will be provided to the student and to the CGNS coordinator. To monitor the integrity of the assessment process and to ensure reasonable equity in assessment workload, the CGNS coordinator will arrange for the assessment to be reviewed by CGNS faculty before the assessment document is issued to the student.

Scheduling and Deadlines

Scheduling of the candidacy assessment is flexible in order to minimize disruption to students’ research programs. Exact dates will be arranged by area faculty in consultation with students. The assessment is usually completed over a period of 8-12 weeks, determined in consultation with the advisory committee and taking into account the student’s other commitments during the assessment period. The oral defense is normally scheduled 1-2 weeks after the assessment documents have been submitted to the examiners.

Evaluation

Candidacy assessment documents submitted for evaluation must be prepared independently and without collaboration. Students are allowed to seek feedback on their writing mechanics (e.g., grammar and sentence construction) from the university writing centre. However, students are not allowed to seek feedback from other sources (e.g., other students or faculty with expertise in the subject area). Successful completion of both components of the candidacy assessment is a requirement of the College of Graduate Studies for the Ph.D. degree. Assessments will be evaluated on a pass/fail basis by the student’s advisory committee. A student who fails either component of the candidacy assessment may be granted a second opportunity to complete that component. In the case of a rewrite, all procedures of the re-examination must be completed within four months of the first exam. However, students may not attempt to pass either component of the candidacy assessment more than twice. 

Candidacy assessment composed of two grant proposals.

Your assessment will consist of the preparation and defense of the equivalent of 2 NSERC grant proposals (i.e., 2 public summaries, 2 X 10pp double spaced proposals, a full reference section for all cited material):

  1. One that represents your primary research program and readings (i.e., the “depth” component).  This should include some description of most of your PhD research (planned, in progress, and completed), and future research that adds to this research program.
  2. One that represents a program of research that demonstrates your knowledge of the secondary readings (i.e., the “breadth” component).

You should use section headings wherever possible, and it is okay to combine these “depth” and “breadth” components as you wish. 

You may include a total of 4 “research contributions” (i.e., papers you have published, submitted, or that are in preparation) along with your grant proposals (these can be helpful when you need to refer the reader to a more detailed description of methodology, etc.).

Your advisory committee will evaluate your proposals at a mutually-agreed-to-time, and schedule an oral defense during which you will receive feedback.

How to Prepare a Winning Proposal

No amount of care and effort in preparing a grant proposal will compensate for a weak research program. However, a poorly prepared proposal can prevent a strong research program from being supported. The following items are important in preparing a Discovery Grant application for NSERC.

The research proposal

Remember that the intent of the NSERC Discovery Grants program is to fund a research program (with long-term goals) rather than a single short-term project or a collection of projects. Thus, a good research proposal will:

  • place the research within the context of what is currently happening in the field;
  • summarize relevant prior work in the field;
  • articulate the goals;
  • describe a research plan and methodology;
  • indicate why the research is useful or important; and
  • provide a progress report on work accomplished with previous funding.

You need to convince the selection committee that:

  • your research program promises a notable advancement or innovation in the discipline or results of importance to a broad range of applications;
  • you have identified well-formulated short- and long-term goals;
  • attaining these goals would be a significant contribution to the discipline;
  • you have a good chance of attaining the goals with the resources available.

A major portion of the proposal should be devoted to a careful description of the research objectives and of the methodology that will be used. For the research plan, you should at least know how you are going to start out and have some ideas for future options.

Applications are judged according to the following criteria. The onus is on the applicants to address these explicitly in their proposal:

  • scientific or engineering excellence of the researcher(s);
  • merit of the proposal;
  • contribution to the training of highly qualified personnel;
  • need for funds.

Describing your ideas

Your proposal will be evaluated by experienced researchers. It is up to you to provide the information necessary for a positive decision.

Acknowledge difficulties honestly. If there are potential problems, say so. It is reasonable to assume that you have thought through your proposal more thoroughly than the reviewers have; consequently, if they see problems that you do not seem to have noticed then they may view your proposal negatively.

In writing a research proposal, you have to address two audiences: 1) the internal and external reviewers, who are likely to be knowledgeable in your field of interest, and 2) the remainder of the committee, who are in your discipline but may have limited knowledge of the area in which you are working. Your proposal must have something for both audiences; there should be enough depth and detail to satisfy the expert, but you must also convince the non-expert of the importance and impact of your proposed research.

Example Reading List

Visual Perception & Motor Control

  • Milner, A.D., & Goodale, M.A. (2006).  The Visual Brain in Action.  2nd Ed. Oxford: Oxford University Press.

Attention

  • Treisman, A. (2004).  Psychological Issues in Selective Attention, in M.S. Gazzaniga (ed.) The Cognitive Neurosciences III, 527-544.
  • Klein, R. (2004).  Orienting and Inhibition of Return, in M.S. Gazzaniga (ed.) The Cognitive Neurosciences III, 545-560.
  • Freiwald, W.A. & Kanwisher, N.G. (2004). Visual Selective Attention: Insights From Brain Imaging and Neurophysiology, in M.S. Gazzaniga (ed.) The Cognitive Neurosciences III, 575-588.
  • Tipper, S.P. (2004).  Attention and Action, in M.S. Gazzaniga (ed.) The Cognitive Neurosciences III, 619-630.

Memory  

  • Davachi, L., Romanski, L.M., Chafe, M.V. & Goldman-Rakic, P.S. (2004). Domain Specificity in Cognitive Systems, in M.S. Gazzaniga (ed.) The Cognitive Neurosciences III, 665-678.
  • Squire, L.R., Clark, R.E., & Bayley, P.J. (2004).  Medial Temporal Lobe Function and Memory, in M.S. Gazzaniga (ed.) The Cognitive Neurosciences III, 691-708.
  • Buckner, R.L. & Schacter, D.L. (2004).  Neural Correlates of Memory’s Successes and Sins, in M.S. Gazzaniga (ed.) The Cognitive Neurosciences III, 739-752.

Consciousness

  • Chalmers, D.J. (2004).  How Can We Construct a Science of Consciousness? in M.S. Gazzaniga (ed.) The Cognitive Neurosciences III, 1111-1120.
  • Crick, F.C. & Koch, C.  (2004).  A Framework for Consciousness, in M.S. Gazzaniga (ed.) The Cognitive Neurosciences III, 1133-1144.
  • Goodale, M.A. (2004). Perceiving the World and Grasping It: Dissociations Between Conscious and Unconscious Visual Processing, in M.S. Gazzaniga (ed.) The Cognitive Neurosciences III, 1159-1172.

Lexical/Semantic Processes

  • Rastle, K. (2007).  Visual Word Recognition, in M.G. Gaskell (ed.) The Oxford Handbook of Psycholinguistics, 71-88.
  • Seidenberg, M.S. (2007).  Connectionist Models of Reading, in M.G. Gaskell (ed.) The Oxford Handbook of Psycholinguistics, 235-250.
  • Ullman, M.T. (2007).  The Biocognition of the Mental Lexicon, in M.G. Gaskell (ed.) The Oxford Handbook of Psycholinguistics, 267-288.
  • McNamara, T.P. (2005).  Semantic Priming. New York: Psychology Press. (200pp.)

Embodied Cognition

  • Klatzky, R.L. & Wu, B. (2008).  The Embodied Actor in Multiple Frames of Reference, in R.L. Klatzky, B. MacWhinney & M. Behrmann (eds.) Embodiment, Ego-Space, and Action, 145-178.
  • Cisek, P. (2008).  The Affordance Competition Hypothesis: A Framework for Embodied Behavior, in R.L. Klatzky, B. MacWhinney & M. Behrmann (eds.) Embodiment, Ego-Space, and Action, 203-246.
  • Culham, J.C., Gallican, J., Cavina-Pratesi, C. & Quinlan, D.J. (2008).  fMRI Investigation of Reaching and Ego Space in Human Superior Parieto-Occipital Cortex, in R.L. Klatzky, B. MacWhinney & M. Behrmann (eds.) Embodiment, Ego-Space, and Action, 247-274.

fMRI Methodology

  • Sarty, G.E. (2007).  Computing Brain Activity Maps. Cambridge: Cambridge University Press.

Evolution and modularity of mind

  • Marcus, G (2008). Kluge: The Haphazard Construction of the Human Mind. Boston: Houghton Mifflin.

Cognitive science candidacy assessment composed of

  1. an examination by content area and
  2. a comprehensive essay in the primary research area.

There are two components to the candidacy assessment. These are 1) a take home examination (due date) and 2) a comprehensive essay (due date).

Part 1: Breadth Examination by Content Area

The breadth component is comprised of five essay questions. However, students who have taken a graduate course in a given content area (i.e., memory; reasoning & problem-solving; perception & attention; or language) are exempt from completing that part of the exam. Each essay should be 5-7 double-space typed pages (1000-1500 words). The questions are intended to provide an opportunity to demonstrate your knowledge and ability to integrate material within each of the major content areas of cognitive science. Each answer will be graded on a pass/fail basis by two faculty readers. To complete the breadth examination, all essays must receive a passing grade from both readers. A failed essay that is judged to be close to a pass may be attempted a second time.

In preparing your essays, please consider the following guidelines:

  1. Each essay should take a position on the question and, in doing so, create, defend, or critique some fairly general theoretical point.
  2. An informative title for each of your essays is appropriate.  Use subheadings where appropriate to help create a clear organizational structure within each of your essays.
  3. The essays should highlight your knowledge of specific theories, experimental paradigms, and phenomena within cognitive science.
  4. The content of the essay should be supported by a substantial reference list that provides up-to-date authority for your claims and arguments. There is no fixed length for each reference list, but about 10 substantially non-overlapping papers, chapters, or books could be sufficient.
  5. The answers submitted must be written independently and without collaboration.

Essay Questions

  1. General Cognitive Science
    Please answer one or the other of the following two general cognitive science questions:
     
    1. Outline and defend what you think should be the primary scientific/theoretical goals of cognitive psychology within the larger domain of cognitive science. Be specific about a) what you think the most important or central theoretical issues are and b) about whether we can in principle resolve these issues using the methods of cognitive psychology. In your answer, be sure to discuss the role of theoretical models in cognitive psychology and how you conceptualize their relation to the "actual mechanisms" of cognition.
    2. Overheard at the Psychonomics conference: “The only future now for cognitive science is brain science, and there is no more useful cognitive psychology without neuroscience.” Do you agree or disagree? In what ways, and to what extent, do you think these statements are true or false?

  2. Memory
    Textbooks on human memory are remarkably uniform in organization and content. From your knowledge of memory research, outline in some detail the contents for a radically “new”  undergraduate course on human memory. It should include theories of concepts and representation.  Provide specific justification for its organization and topic coverage (e.g, what theoretical principles about memory motivate its organization and content?).

  3. Reasoning and Problem Solving
    Defend the proposition that humans are rational.  There is a great deal of evidence to indicate that humans frequently fail to reason in accord with an accepted, normative standard.  Does this evidence indicate that humans are irrational?  Are the normative standards appropriate benchmarks of human rationality?  Why or why not?

  4. Perception and Attention
    Investigations of the mechanisms and processes underlying attention are many and varied in cognitive psychology.  In your view, what are the main components and/or phenomena that any comprehensive model of attention should address?  How do you view the relationship between attention and the distinction between conscious and unconscious processes?

  5. Language
    Broadly speaking, research on the perception, cognition, and production of language involves a variety of levels of process description (e.g., modularity/interactivity, computational goal, representational algorithm, physical implementation) and has resulted in several controversial issues. Select the area of language research that you consider to be the most controversial and (i) explain the issue with reference to the level(s) of process description involved, (ii) briefly describe the research for and against each side of the issue, and (iii) describe your criticisms of the research and suggestions for improvement.

Part 2: Comprehensive Essay

The goal is a broad, integrative position paper concerning your area of primary research.  The supervisor and advisory committee in consultation with the student will identify the scope and subject matter of the essay. The scope of the essay should substantially exceed the scope of your dissertation topic. The essay constitutes an examination, and should be prepared independently and without collaboration. There will be an oral defense of the paper.