Articles > Feature Articles

The Damocles Sword of Academic Publishing

Sharper Students or Duller Sword in the MIS Field?

By Kai R. Larsen

The Damocles Sword of Academic Publishing

Abstract

Doctoral students often find it hard to understand at what level of productivity they should be. Through an analysis of resumés of doctoral students in the Management Information Systems (MIS) field, a better understanding of what is expected of current students as compared to former students is achieved. Both conference presentations and publications in journals are examined. Finally, there is an examination of whether the quantity of publications can be related to the ranking of the school that a student attends.

Introduction

About 350 BC, Damocles, a courtier of Dionysius the Elder (the ruler of the city Syracuse in Sicily), let his envy of the ruler be known. Hearing about Damocles' envy, Dionysius soon thereafter invited Damocles to a luxurious banquet, where the grandeur and happiness of being a ruler became even more obvious. However, while he was enjoying the banquet, Damocles' happiness subsided when he was made aware of the sword that was hanging above his head by a single horsehair. In this way Dionysius taught Damocles that insecurity might threaten those who appear to be the most fortunate [2]. Doctoral students, it seems, can be viewed as becoming increasingly willing to sit below the academic "Damocles sword" by starting their publishing career earlier than was once expected.

In his Communications of the ACM article describing the demographics of candidates for faculty positions in computer science, Anthony Ralston pointed out that "the number of publications on the CVs of new Ph.D.s is impressive, even if almost all of them are written jointly with advisors. When I received my Ph.D. (in mathematics) in 1956, I was proud already to have one publication, since then the first publication for most people was the post-degree publication of their dissertation research" [4]. There are two basic goals of the current analysis: to find out whether the pattern Ralston speaks of is found in candidates for faculty positions in the Management Information Systems (MIS) field, and to examine the publication outlets for students in this field while at the same time comparing the availability of these outlets to the students. Thus, this study was conducted partly to analyze existing trends, and partly as a guide for new students in the MIS field.

Management Information Systems (MIS) as a field is comprised of students and faculty in MIS departments of business schools. It also includes people in interdisciplinary programs across a number of departments (e.g., Computer Science, Business, Geography, Public Administration, Communication, Information Science, and Sociology). Most MIS doctoral students are members of ACM and some Computer Science doctoral students end up working as faculty in the MIS field, either through cooperation with faculty at the Business School, or through employment in the Business School. Also, there seem to be good opportunities for computer science students to enter the MIS field. In the 1997/98 recruiting year, there were about two positions available in MIS at the entry level for each doctoral student that had declared an interest in these positions.

Depending on the specific institution at which a student applies for a faculty position, emphasis will be placed on different elements of their experience. Among the factors that will receive the most weight are: publications, presentations, teaching experience, working papers, reputation of institution, reputation of faculty on dissertation committee, references, the network and contacts of both the applicant and faculty members with whom the applicant works, personality, service to own institution, job responsibilities, and professional experience. It is believed that the volume and quality of a student's publications are among the most important factors in securing an on-campus interview. When this interview has been granted, the experience gathered from doing presentations at conferences may further positively affect a candidate's chances. Publishing is important not just because it enhances the possibility of employment, but also because it increases understanding of the publishing process. After gathering knowledge about this process, the student may decide that a tenure track position is not as desirable as he had earlier believed. The student may then turn his attention to elements of the doctoral program that will enhance the probability of securing a good position in industry or government. Otherwise, the student may continue the process of preparing for a successful career in academia.

A problem with which students are often faced at the beginning of their stay in a doctoral program is the lack of absolute standards. While many programs have rigid requirements of specific courses and comprehensive exams, doctoral students whose goals are to become a faculty member in their field face the problem of understanding the unwritten rules about what attracts recruiters. Because of the constant fluctuation in demand for faculty members in a given field, students often begin their studies without the knowledge needed to increase their possibilities for later employment. By the time students who do not have the benefit of good advice from their fellow students or faculty advisors reach the point where they understand whether publishing is important for their career, it may be too late to catch up. The remainder of this discussion is dedicated to the improvement of that understanding.

Research Methods and Descriptive Information

The data was gathered at the AIS-ICIS placement Website. This site allows students interested in faculty positions in the MIS field to register their resumé. By examining student resumés, publication patterns were found. To ensure that the most desirable candidates would not remove their resumés, data was gathered at two points: late in the Fall semester of 1997 and the Spring semester of 1998. Although only five student resumés were unavailable in 1998, a number of new resumés had been added, enhancing the richness of the data.

Data for 147 students was gathered. Of these, 40 were excluded from the study because they had received their Ph.D. or D.B.A. at an earlier point in time; including their publications would have skewed the results. Only students that received, or expected to receive, their doctoral degrees after August 1997 were included. Because of the desire to understand the publication pattern of students at American universities, students with degrees from foreign institutions were also removed from the pool. It is, however, interesting to note that among the 14 international students removed, there were some excellent candidates with very impressive publication records and degrees from renowned universities such as the University of Oxford and the London School of Economics and Political Science. Of the five students that were not available in the spring of 1998, data from 1997 were used for three of them, and two students were excluded from the study. The data used for the study were the resumés of the remaining 91 students who expected to receive their doctoral degrees from U.S. universities sometime after August 1997.

To examine whether rankings of universities and schools has any effect on student productivity, two rankings were used: the Carnegie rankings of universities, which is a ranking system that is focused on the whole university's federal funding and number of doctorates awarded annually, and the recent US News and World Report rankings of business schools. The Carnegie ranking was not expected to correlate with the number and quality of student publications. A 'Research University I' "gives high priority to research"; they award 50 or more doctoral degrees each year, and receive annually $40 million or more in federal support. A 'Research university II' also awards 50 or more doctoral degrees each year, and receives between $15.5 million and $40 million in annual federal support. A 'Doctoral University I' awards at least 40 doctoral degrees annually, whereas a 'Doctoral University II' awards at least ten doctoral degrees in three or more disciplines, or 20 or more doctoral degrees in one or more discipline [1]. This information is included to provide a more complete picture of the universities at which these students have received or will receive their degrees. From this information, it can be gathered that most students get their degrees from large research universities. Comparing these data with the number of students within each category [1], the division of students in this sample is fairly representative of the general division of students among the categories, with the exception of 'Doctoral II' institutions, who predictably have a lower doctoral student to other student ratio.

Table 1. Carnegie Rankings of Students' Institutions (percentages rounded)

Research University I

51 students (56%)
Research University II

15 students (17%)
Doctoral University I

22 students (24%)
Doctoral II

3 students (3%)

Because the Carnegie rankings were expected to have little predictive value, the recent US News & World Report ranking of business school specializing in management information science was used [5]. The report ranks what it considers to be the top 30 business schools in the field. This was used to rank the doctoral programs in this study into four tiers. Those ranked one to ten constituted tier one, whereas tier two consisted of the programs ranked eleven to twenty. Tier three consisted of programs ranked twenty-one through thirty, and tier four was reserved for programs or business schools not ranked in US News & World Report. It is interesting to note that 60% of the students belonged to programs not ranked by the report. This, however, should not be taken to mean that these programs are necessarily of lesser quality than the ranked programs. Some students were educated in programs not directly located within the business school, such as the University at Albany, S.U.N.Y. 's interdisciplinary Information Science Ph.D. program, where the students take courses within a number of schools and departments, some of which are highly ranked in the same US News & World Report study [5].

Table 2. US News & World Report Ranking of Students' Schools

Tier I

17 students (19%)
Tier II

8 students (9%)
Tier III

11 students (12%)
Tier IV

55 students (60%)

When examining students' publications, all conference presentations will be referred to as publications, regardless of whether the conference published proceedings, and whether the students' papers were published in those proceedings. Because the data are reported by the students themselves, their accuracy is hard to verify. In some cases the data provided by students were compared against real conference or journal publications to verify accuracy. Some of the data was found not to be up-to-date.  Though one would expect the students looking for a position to update their resumé frequently, several resumés were clearly not updated. A case in point was the student claiming that he expected to receive his doctoral degree in August 1997. His resumé had not been updated six months after that date. Also, one student listed his submitted presentation to ICIS9297—a paper that was not found in the proceedings—more than a six months after the student must have been notified that the paper was not accepted. Students have much to gain by trying to look as professional and accurate as possible; at the same time, the students have much to lose by reporting false data. Because of these factors, the data is expected to be fairly accurate. Journal and conference presentations that were listed as "submitted" were not counted; however, if they were listed as accepted and forthcoming, they were included in the study. Technical reports and unpublished cases were removed from the study along with doctoral student consortium presentations.

Results and Discussion

Though no comparative data exists, doctoral students in the MIS field seem to be quite productive. Among the 91 students, there were 119 journal publications and 379 conference presentations, most of which were co-authored. The students also had a total of six books and fifteen book chapters. The average student had 1.3 journal publications and 4.2 conference presentations. One out of every six students had published a book chapter. An overview is presented in Table 3.

Table 3. Basic Statistics (ranked entries are subsets of unranked entries)

  Mean Median St. Dev. Sum
Books 0.06 0.00 0.29 6
Chapters 0.16 0.00 0.43 15
Journals 1.31 1.00 1.64 119
Conferences 4.16 3.00 3.58 379
Ranked Journals 0.38 0.00 0.77 35
Ranked Conferences 1.66 1.00 2.03 151
Ranked Conferences + AIS 2.49 2.00 2.50 227

To examine whether students just publish in journals and conferences outside of the field, or in outlets that are extremely easy to gain acceptance to, the student publications were cross-referenced against a ranking of journals and conferences in the MIS field. Hardgrave and Walstrom [3] ranked the publication outlets in the field based on a wide survey of faculty members in the field. The three rows that start with 'ranked' display only the student publications accepted for such outlets. The last row displays the ranked conference publications, but adds the student publications in the Association for Information Systems Americas Conference. This turned out to be a major outlet for student publications, and was not included in the Hardgrave and Walstrom study because "at the time of [the] study, AIS had yet to have its first meeting" [3]. The difference between the median and the mean for these statistics indicates that the mean is "pulled up" by a few extremely productive students.  

To explore whether the number of student publications correlates with the US News and World Report ranking of business schools, the Pearson correlation coefficient was calculated (Table 4). It is interesting to note that the only significant correlation was found between ranking and journals. The higher the ranking (closer to number one), the higher the number of journal publications. This correlation, though, was very low—too low to draw any conclusions. Interestingly enough, this correlation disappears when only considering journals that are ranked in the MIS field. To further examine the publication productivity, Figure 1 shows at what percentile a particular student would be based on the student's number of publications. The 'ranked conferences' include AIS Americas Conference publications.

Another interesting item to consider is the number of student publications in ranked and recognized MIS journals. One would expect the student publications to crowd the bottom half of the scale, but this is not the case.  Hardgrave and Walstrom listed 53 ranked journals in their study; the mean ranking of student publications was about 13. With a median of 10 and a standard deviation of 11, it seems quite clear that when students publish in these journals, they publish in those that are highly ranked. It is unknown whether the other journals the students were found to publish in are of similar quality in other fields. Though the answer to that question is hard to find, one student paper was found to have been published in a top journal in the nursing field, Nursing Research.

The papers on which students collaborated had up to six authors, but three was the most common number of authors, followed by two. Among the papers where the student's author-position was found (22 out of 35), it was most common that the student was the second or first author. However, students that are first or second authors of papers are probably more likely to list their publications in correct order, whereas students that are among the peripheral authors are more likely to list all other authors at the end of the reference (usually saying `with' and then listing the other authors, thereby making data interpretation more difficult). Finally, the most common outlets for student conference publications are displayed in Table 5.

Table 5. Popular Conferences among Student Scholars

Conference Name Ranking in Hardgrave and Walstrom Number of student publications (of 379)
AIS Americas Conference Not ranked 75
Annual Decision Sciences Institute  5 62
INFORMS Annual Conference 8 18
Regional Decision Sciences Institute  11 17
ICIS 1 15
IRMA International Conference 9 14
Hawaii International Conference on System Sciences 2 13
Academy of Management Annual Conference 10 8
Sum   222

It is very interesting to note from Table 5 that although the Regional Decision Sciences Institute is a very popular outlet for student scholars, the relatively new AIS Americas Conference has taken over the lead. Though the AIS conference only started in 1995, it already has more student presentations than any other conference examined. It would seem that the informal nature of the conference, coupled with its reasonably inclusive publication standards, have made it the preferred conference for many doctoral students.

Conclusions

When analyzing the conference publications, it is clear that more than one third of all conference publications are divided between two conferences, the AIS Americas Conference and the annual DSI conference. Also, it was not possible to find any clear connection between the number of publications and the ranking of schools.

Though the current hiring situation for students in the MIS field is very favorable, with more positions than candidates to fill those positions, the students are not resting on their laurels. Students are preparing themselves for their coming positions as researchers by publishing in a number of outlets. It seems that the "publish or perish" attitude has reached the students of our institutions. Many Doctoral students are publishing in well-respected journals and conferences.

It is interesting to note that the students studied in this research did about 70 percent of their publishing outside of the traditional MIS journals and conferences. This attests to the highly multidisciplinary nature of the field in addition to the wide acceptance of MIS students' studies in other fields.

If we accept Ralston's notion that students are publishing more and earlier than was once expected, the question becomes whether students are getting better at publishing and do more interesting research, or whether the conferences and journals available have lowered the standards necessary to publish. Only more research can clearly answer this question.

Rerferences

1. Boyer, Ernest L., A Classification of Institutions of Higher Education 1994. Carnegie Foundation, December 1994

2. "Damocles," Microsoft Encarta 98 Encyclopedia. Microsoft Corporation, 1998

3. Hardgrave, Bill C., and Walstrom, Kent A, Forums for MIS Scholars, Communications of the ACM, 40, 11 (November 1997), pp. 119-124.

4. Ralston, Anthony, The Demographics of Candidates for Faculty Positions in Computer Science, Communications of the ACM, 39, 3 (March 1996) pp. 78-84.

5. U.S. News and World Report, Americas Best Graduate Schools 1998 : Plus a Directory of Law, Business, Medical and Engineering Schools, U S News Specialty Marketing, 1998

Biography

Kai R. Larsen (klarsen@acm.org) is a student at the University at Albany, S.U.N.Y.'s Information Science Ph.D. program. He specializes in information decision systems and organizational studies and is currently working on  network organizations and their potential as organizing forms for interorganizational information systems development.

© Copyright 1998 Kai R. Larsen

This article was contributed by our friends at Crossroads, a publication of the Association of Computing Machinery.

computer sciencegraduate school



Articles > Feature Articles

newletter