[Federal Register: May 9, 1997 (Volume 62, Number 90)]
[Proposed Rules]               
[Page 25578-25582]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[DOCID:fr09my97-35]

=======================================================================
-----------------------------------------------------------------------

POSTAL RATE COMMISSION

39 CFR Part 3001

[Docket No. RM97-2; Order No. 1174]

 
Amendment to Rules Concerning Evidence Based on Market Research

AGENCY: Postal Rate Commission.

ACTION: Notice of proposed rulemaking.

-----------------------------------------------------------------------

[[Page 25579]]

SUMMARY: The Commission proposes to amend its rules of practice by 
clarifying foundational requirements for market research evidence. The 
amendment entails three substantive revisions. One requires a survey 
sponsor to provide additional supporting information about technical 
aspects of the market research. Another provides for participants using 
statistical techniques to limit the possibility of disclosing the 
identity of a survey respondent and data uniquely associated with that 
respondent. The third clarifies the level of access to data files and 
computer programs that is to be provided, including the stage at which 
rights to replication of survey results attach. These revisions will 
clarify rule 31(k)(2)'s applicability to market research, thereby 
reducing the need for case-by-case determinations and minimizing the 
potential for delay in issuing Commission recommendations. The 
amendment also makes minor editorial improvements in rule 31(k).

DATES: Comments responding to this notice of proposed rulemaking must 
be submitted on or before June 9, 1997.

ADDRESS: Comments and correspondence should be sent to Margaret P. 
Crenshaw, Secretary of the Commission, 1333 H Street NW, Suite 300, 
Washington, DC 20268-0001 (telephone: 202/789-6840).

FOR FURTHER INFORMATION CONTACT: Stephen L. Sharfman, Legal Advisor, 
Postal Rate Commission, 1333 H Street NW., Suite 300, Washington, DC, 
20268-0001, (telephone: 202/789-6820).

SUPPLEMENTARY INFORMATION: Rule 31(k) outlines foundational 
requirements for studies and analyses offered as evidence to be relied 
upon in Commission proceedings. See 39 CFR 3001.31(k). Submissions 
within the rule's purview are subject either to the terms of rule 
31(k)(1) or to similar provisions tailored to specific types of 
statistical studies and computer analyses. See generally rule 31(k)(2) 
and (k)(3) (39 CFR 3001.31(k)(2) and (k)(3)). These requirements call 
for a description of the study plan, relevant assumptions, data 
collection techniques, the facts and judgments upon which conclusions 
are based, alternative courses of action considered, and certain other 
supporting information and data. This material must be filed along with 
the related study or be produced upon request.
    Rule 31(k) generally reflects settled evidentiary principles, but 
persistent questions have arisen in recent proceedings about the impact 
of certain market research practices on the interpretation of 
paragraphs (k)(2) and (3). The debate has centered primarily on three 
major--and often interrelated--concerns. One is the interest survey 
sponsors assert in providing survey participants with reasonable 
assurances that their participation in the survey and the sensitive 
commercial data or information provided in their responses will not be 
disclosed. Another is reviewers' interests in replicating survey 
results and, in certain instances, in using a preferred method to 
accomplish that end. A third issue is the impact of computer-assisted 
data collection (CADC) techniques on compliance with rule 31(k)'s 
requirement that ``input data'' be provided. See PRC Op. MC93-1, paras. 
117-122; see also PRC Op. MC95-1, Appendix C. CADC techniques, in 
particular, have altered some participants' expectations about how--and 
whether--rule 31(k)'s data disclosure requirements apply to certain 
market research efforts.
    The Commission has resolved conflicts on a case-by-case basis, but 
finds that revising the rule to provide participants with additional 
guidance on how market research submissions should be supported is 
warranted. PRC Op. MC95-1, Appendix C at 1-2. Having had an opportunity 
to review pertinent issues and concerns outside the constraints imposed 
by motion practice, the Commission has made preliminary determinations 
about the manner in which rule 31(k) should be revised.

Postponement of Comprehensive Revisions

    While developing an amendment to address problematic aspects of 
market research submissions, the Commission also considered proposing a 
comprehensive reorganization of rule 31(k). Structurally, a 
comprehensive review would permit redundant or overlapping requirements 
to be modified or eliminated, thereby simplifying a rule that has been 
the subject of several amendments. In addition, consideration could be 
given to whether the numerous provisions now covered in paragraph (k) 
should continue to be located within rule 31, which is an umbrella 
evidentiary provision, or whether they should form an independent 
provision.
    Substantively, a broader focus would provide an opportunity to 
address the advisability of maintaining certain formal distinctions 
within the rule, such as a special set of requirements for computer-
based studies. Computer-specific provisions were added to the rule (in 
subparagraph (3)) in the early 1980s, when the use of computers to 
prepare or develop evidence was in its infancy and several related 
evidentiary issues were unsettled. Since then, computer use has become 
routine not only for submissions covered by rule 31(k)(3), but in the 
preparation of nearly every filing in Commission proceedings. Thus, the 
rule's underlying orientation may warrant reconsideration. At a 
minimum, revisions to subparagraph (3) and conforming changes in other 
provisions may need to be made. See generally Docket No. RM81-1, Notice 
of Proposed Rulemaking (NPRM) at 5. Similarly, the need for 
distinctions between studies that involve statistics and those that do 
not could be reviewed. Moreover, a broader approach might allow issues 
related to the emergence of electronic data bases, from which a number 
of different studies and analyses can be developed, to be explored.
    The Commission's interest in updating and reorganizing the rule is 
tempered with a concern that wholesale revision might unduly delay 
addressing the questions that have surfaced about market research. 
Based on this consideration and an assessment that other aspects of the 
rule appear to be working reasonably well, the Commission has decided 
to propose only limited changes now. Accordingly, the proposed 
amendment has been drafted to conform, as closely as possible, to the 
existing approach and to cause minimal disruption to the current 
numbering system. Structurally, this is accomplished primarily by 
distinguishing market research from other sample surveys, with the 
requirements specific to market research designated as rule 
31(k)(2)(i). Existing rule 31(k)(2)(i), now entitled ``Sample 
surveys,'' is renamed ``Other sample surveys'' and redesignated as rule 
31(k)(2)(ii). Conforming numbering changes are also made to other 
subparagraphs of the rule. Substantively, the Commission notes that 
this amendment is not inconsistent with its recent statement, in PRC 
Op. MC96-3, that the existing rules on sample surveys require certain 
quantitative disclosures. See generally PRC Op. MC96-3 at 37-38. Given 
its general position on the scope of the existing sample survey 
requirements, the Commission is not proposing to make them more 
explicit at this time.

Expanded Foundational Requirements for Market Research-Based 
Submissions

    Rule 31(k) now provides, in subparagraph (2)(i)(a), that a 
proponent of a sample survey is to provide a clear description of the 
survey design, including the definition of the universe

[[Page 25580]]

under study, the sampling frame and units, and the validity and 
confidence limits that can be placed on major estimates. The rule also 
provides, in subparagraph (2)(i)(b), that the survey sponsor provide an 
explanation of the methods of selecting the sample and the 
characteristics measured or counted.
    These provisions generally provide a straightforward and 
serviceable base for evaluating sample surveys. However, given the 
growing importance of market research in Commission proceedings, it 
appears advantageous to be more specific about the detail sponsors 
should provide at the time the market research (or the submission it 
supports) is filed. This also is consistent with the Commission's view, 
previously expressed in connection with survey replication, that 
providing descriptions of technical procedures can provide reviewers 
with the ability to make an assessment, from the description itself, of 
the appropriateness of various standard procedures. See PRC Op. MC95-1, 
Appendix C at 7.
    The Commission therefore proposes to specify, in 
Sec. 3001.31(k)(2)(i)(a)(1), that the foundational requirements include 
details of the sampling, observational, and data preparation designs, 
with definitions of the target population, sampling frame, units of 
analysis and survey variables. These requirements also include an 
explanation of the methodology for the production and analysis of the 
major estimates and the associated sampling errors. Proposed 
Sec. 3001.31(k)(2)(i)(a)(2) requires that the proponent not only 
provide measures of sampling error, but also present coverage, response 
and editing rates.
    In addition to these changes, which are primarily adaptations of 
existing requirements for sample surveys, the Commission is also 
proposing four new foundational requirements. Proposed subparagraph 
(a)(3) requires a discussion of data comparability over time and with 
other data sources, and the effects of benchmarking and revisions. 
``Benchmarking,'' in this context, refers to establishing an acceptable 
standard by which to evaluate estimates. Subparagraph (a)(4) requires 
an assessment of the effects of editing and imputation and other 
potential sources of error on the quality of the survey estimates. 
Subparagraph (a)(5) requires identification of applicable statistical 
models when model-based procedures are employed. Finally, subparagraph 
(a)(6) requires an explanation of all statistical tests performed and 
an appropriate set of summary statistics summarizing the results of 
each test.

Confidentiality

    Confidentiality issues have dominated recent motion practice, and 
they have been a concern for some time. In Docket No. RM81-1, for 
example, the American Bankers Association (ABA) filed comments noting 
that compliance with a requirement of producing actual input data upon 
request could pose difficulties because of confidentiality promises. In 
evaluating ABA's position, the Commission noted that the provision in 
question was not a new element of the proposal under consideration, but 
had been in effect for several years without creating serious 
difficulties. The Commission concluded, at that time, that it preferred 
to continue its practice of addressing special needs for 
confidentiality as they arose, rather than alter the general rule to 
meet exceptional cases. Docket No. RM81-1, Final (Rulemaking) Notice at 
11. In PRC Op. MC95-1, however, the Commission clearly signaled its 
interest in ending this practice by stating that it intended to 
institute a rulemaking and explore whether a widely-applicable standard 
or policy statement on the confidentiality of market research data and 
information could be developed.
    The Commission has made a preliminary determination that the 
continuing motion practice on this topic confirms the need for a 
revision to its existing practice, and that statistical disclosure 
limitation (SDL) methods provide a workable, objective standard. SDL 
methods are techniques that limit the risk of disclosure of individual 
information when statistics are disseminated in tabular or microdata 
formats. These practices are not new, but have been developed and 
implemented by various federal agencies over the past 25 years. See 
generally Jabine, Thomas B., ``Statistical Disclosure Limitation 
Practices of United States Statistical Agencies,'' Journal of Official 
Statistics, Vol. 9, No. 2 (1993) at 427-454.
    In conjunction with this rulemaking, the Commission is 
establishing, for participants' convenience, Library Reference PRC-LR-
1, containing Statistical Policy Working Paper 22, ``Report on 
Statistical Disclosure Limitation Methodology'' (May 1994), (hereafter, 
Working Paper). The report was prepared by the Subcommittee on 
Disclosure Limitation Methodology of the Federal Committee on 
Statistical Methodology, which is associated with the Statistical 
Policy Office of the Office of Information and Regulatory Affairs of 
the Office of Management and Budget. The preface indicates that the 
report includes a tutorial, guidelines, various recommendations, and an 
annotated bibliography. The Commission notes that the report 
specifically indicates that legal questions are beyond its scope. 
Working Paper at 2. An excerpt on survey research from the ``Reference 
Manual on Scientific Evidence,'' published by the Federal Judicial 
Center in 1994, is also included in the library reference.
    The specific provision the Commission proposes adding, in new rule 
31(k)(2)(i)(b) is: ``Protection against disclosure of sensitive data 
should be provided through the application of appropriate statistical 
disclosure limitation (SDL) practices when data are produced for 
secondary analysis.'' The rule indicates that SDL practices include the 
following: Removal of respondent identifiers from microdata files; cell 
concentration and suppression rules; and data masking through 
aggregation, random noise injection, and simulation of artificial 
records.
    In the sense used in the rule, a microdata file consists of 
individual records, each containing values of variables for a single 
person, business establishment or other unit. Id. at 3. There are no 
identifiers on the file, and the data may be disguised in some way to 
ensure that the individual data items cannot be uniquely associated 
with a particular respondent. Id. at 6. Cell concentration means that a 
specific number of cases in a given cell of a data table cannot account 
for a percentage of the cell total equal to or exceeding a prescribed 
threshold; that is, the (n,p) cell concentration rule is violated if n 
or fewer respondents account for at least p percent of the total cell 
value. If this rule is violated, the cell is suppressed or collapsed 
with other cells to reduce the risk of disclosure. Data masking entails 
distorting data prior to its release or limiting the amount of data 
released. It can involve random error (noise) to the data entries, 
multiplying the data by random values from known distributions, or data 
swapping. The latter refers to the practice of interchanging the values 
for survey items of sample cases having similar characteristics or 
values for auxiliary variables.
    The proposed rule also provides that statistical disclosure is 
defined as the identification of the respondent or the linking of a 
respondent to sensitive data in a survey record or data file. The 
revised rule also affirmatively states that under certain conditions, 
the post-SDL data shall be the starting point for an evaluation on the 
merits.
    The Commission recognizes that its endorsement of SDL techniques as 
a means of limiting disclosure

[[Page 25581]]

presupposes certain expertise on the part of both survey sponsors and 
survey reviewers. However, the complexity of the surveys that have been 
at the center of recent motion practice indicate that the sponsor (in 
most cases, the Postal Service) would have access to the resources 
needed to meet the rule's standard. To the extent that survey reviewers 
might need assistance in understanding the discovery implications of 
the SDL techniques, technical conferences or other forms of assistance 
can be made available.

Reviewers' Access to Data, Including Replication of Survey Results

    As with confidentiality, replication of survey results also has 
been an issue over the course of several proceedings. In Docket No. 
RM81-1, for example, the Postal Service questioned whether the word 
``replicate'' in the proposed rule imposed too broad a standard. The 
Commission concluded that it did not, emphasizing that the fact that 
this term was not explicitly used in the final rule reflected a 
technical drafting decision, rather than a substantive change in 
position. The Commission said: ``We think it is clear--even without 
express use of this term--that the final rule allows a participant, 
upon proper request, to obtain materials that would allow replication 
of the results of computer-generated presentations.'' Final 
(Rulemaking) Notice at 13.
    By Docket No. MC95-1 the question of what ``materials'' should be 
made available had come to the fore, and the Commission noted that 
newer market research techniques complicated the issue. It cautioned:

    (P)articipants' insistence on the ability to trace a numerical 
result from CADC market research to the primary data source by 
replicating various data adjustments may often be very impractical, 
and sometimes simply impossible. The task of verifying a specific 
numerical result could, in itself, entail running a rather extensive 
set of complex computer programs associated with the survey's data 
collection, editing, coding, estimation and analysis procedures.

    PRC Op. MC95-1, Appendix C at 6. In a related comment, the 
Commission noted that it viewed the overall objective of the rule as 
``* * * placing reviewers in a position to determine whether the data 
are sufficiently accurate to satisfy the (evidentiary) standards the 
Commission must apply.'' Id.
    The Commission also noted that one difficulty in resolving disputes 
is that several key terms in rule 31(k) are not defined. The Commission 
suggested that consideration should be given to whether inclusion of a 
set of definitions or guidelines for interpretation might be useful. At 
this time, the Commission has decided against including definitions of 
the proposed SDL techniques in the rule, but is clarifying the meaning 
of ``input file.'' In addressing participants'' uncertainty over the 
meaning of this term, the Commission has noted previously that an 
``input file'' can be any data set that is entered into a statistical 
program or package designed for a specific purpose, and that it is 
therefore

    * * * unlikely that the ``raw data'' and the ``input data'' for 
adjustment and estimation programs would be coincidental in a 
moderate-to-large survey research effort. This is because the raw 
data are usually modified to some extent--even if no more than 
recoded--before they are entered in a database.

    PRC Op. MC95-1, Appendix C at 4. To address this situation, the 
Commission proposes a new provision--in rule 31(k)(2)(i)(b)--providing 
that access ``shall be sufficient to permit the replication of 
electronic data processing after production of the edited data file.'' 
The term ``edited data file'' is defined in the rule as raw data after 
appropriate coding, editing for consistency checks and application of 
SDL methodology.''

Availability of Opportunity To Request Waiver

    Assuming adoption of the proposed amendment, the Commission expects 
participants to exercise all reasonable efforts to comply with its 
terms, including the use of SDL methods. To the extent a participant 
believes it cannot do so, but nevertheless seeks evidentiary status for 
affected submissions, a separate rule of practice--rule 22--provides an 
opportunity to seek waiver, in whole or in part, by filing a timely 
motion. See 39 CFR 3001.22. Waiver is conditioned on a showing that the 
interests of other participants will not be unduly prejudiced, and that 
it is consistent with the public interest and the Commission's 
discharge of its responsibilities. Given these conditions, the 
Commission expects that a participant seeking relief from application 
of the new requirements would propose, at a minimum, alternative means 
of satisfying the interests sought to be protected by rule 31(k).

Limited Editorial Improvements

    The Commission is also proposing limited editorial improvements in 
Sec. 3001.31(k)(3) at this time. One entails the proposed deletion of a 
citation to outdated software documentation standards. These standards 
were current when the related text was added to rule 31(k) in the early 
1980s, but are now seriously outdated and, in some instances, out of 
print. The Commission has considered, but rejected, replacing these 
references with more current standards, on the assumption that 
participants no longer need to be provided with examples of 
documentation. Thus, the proposed amendment eliminates the footnote 
citation associated with the word ``standards'' in the main text of 
rule 31(k)(3)(i)(e) and deletes the related footnote in its entirety. 
The Commission also considered replacing the reference to ``magnetic 
tape'' in rule 31(k)(3)(i)(i) with a more generic term or phrase, but 
instead decided to change it to ``a compact disc.'' The Commission 
invites comments on retaining the reference in the same provision to a 
time-sharing service.

Comments

    To assist commenters in preparing a response to this proposal, the 
Commission reiterates its conscious decision to keep the focus of this 
rulemaking comparatively limited. Thus, this proposal addresses the 
existing sample survey provisions only in the sense of their 
application to market research. Within this framework, commenters are 
invited to submit comments addressing pertinent issues. In particular, 
the Commission welcomes attention to the following matters:
    <bullet> Whether participants anticipate difficulties in employing 
SDL methods and, if so, what these might be;
    <bullet> Whether participants are aware of any supplementary 
methods or approaches that could or should be included in the rule;
    <bullet> Whether the general availability of an opportunity to 
request waiver under rule 22 is sufficient, or whether waiver should be 
further conditioned or restricted through express language in rule 
31(k)(3);
    <bullet> Whether the definitions in this Notice of Proposed 
Rulemaking provide participants with sufficient information on SDL 
techniques;
    <bullet> Whether the Commission's assumption that a reference to 
specific software standards is no longer needed is correct, or whether 
the standards should be updated; and
    <bullet> Whether other minor editorial revisions in rule 31(k) are 
necessary or desirable at this time, and can be incorporated with 
minimal disruption.

List of Subjects in 39 CFR Part 3001

    Administrative practice and procedures, Postal Service.

[[Page 25582]]

    For the reasons set forth in the preamble, 39 CFR Part 3001 is 
amended as follows:

PART 3001--RULES OF PRACTICE AND PROCEDURE

    1. The authority citation for 39 CFR part 3001 continues to read as 
follows:

    Authority: 39 U.S.C 404(b), 3603, 3622-24, 3661, 3662.

    2. 39 CFR 3001.31(k) is amended as follows:
    3. Redesignate paragraph (k)(2)(i) through (iii) as (k)(2)(ii) 
through (iv).
    4. Amend redesignated paragraph (k)(2)(ii) by changing the title 
from Sample surveys to Other sample surveys.
    5. Add paragraph (k)(2)(i) to read as follows:


Sec. 3001.31  Evidence

* * * * *
    (k) Introduction and reliance upon studies and analyses--(1) * * *
    (2) * * *
    (i) Market research. (a) The following data and information shall 
be provided: (1) A clear and detailed description of the sampling, 
observational, and data preparation designs, including definitions of 
the target population, sampling frame, units of analysis, and survey 
variables;
    (2) an explanation of methodology for the production and analysis 
of the major survey estimates and associated sampling errors;
    (3) a presentation of response, coverage and editing rates;
    (4) a discussion of data comparability over time and with other 
data sources, and the effects of benchmarking and revisions;
    (5) an assessment of the effects of editing and imputation and 
other potential sources of error on the quality of the survey 
estimates;
    (6) identification of applicable statistical models, when model-
based procedures are employed;
    (7) an explanation of all statistical tests performed and an 
appropriate set of summary statistics summarizing the results of each 
test.
    (b) Upon request, access to data files and computer programs shall 
be provided. Access shall be sufficient to permit replication of 
results after development of the edited data file. For purposes of this 
subparagraph, the phrase ``edited data file'' refers to raw data after 
appropriate coding, editing for consistency checks and application of 
statistical disclosure limitation methods (SDL) methods.
    (c) Protection against disclosure of confidential commercial data. 
(1) If the recipient of a request for data pursuant to this paragraph 
asserts that compliance with the request would conflict with a 
confidentiality agreement, the recipient shall be expected to employ 
SDL methods to protect against the disclosure of confidential 
commercial data. The SDL method(s) selected shall not interfere with 
other reasonable or expected uses of the data.
    (2) For purposes of this subparagraph, SDL methods include the 
removal of respondent identifiers from microdata files; cell 
concentration and suppression rules; and data masking through 
aggregation, ``random noise'' injection, and simulation of artificial 
records. Statistical disclosure means the identification of the 
respondent or the linking of a respondent to sensitive data in a 
tabular presentation, survey record or data file.
    (3) If the results or conclusions reached after application of the 
SDL method(s) differ materially from those reached prior to such 
application, the post-SDL data shall be deemed controlling for purposes 
of the sponsoring party's evidentiary presentation and related legal 
argument.
    6. Revising paragraph (k)(3)(i)(e) to read as follows:
* * * * *
    (k) * * *
    (3) * * *
    (i) * * *
    (e) For all source codes, documentation sufficiently comprehensive 
and detailed to satisfy generally accepted software documentation 
standards appropriate to the type of program and its intended use in 
the proceeding.
    7. Revise the first sentence of the concluding text after paragraph 
(k)(3)(i)(i) to read as follows:
* * * * *
    (k) * * *
    (3) * * *
    (i) * * *
    (i) * * *
    Paragraphs (k)(3)(i)(d) and (f) of this section shall be provided 
either in the form of a compact disc or through access to a time-
sharing service, at the option of the provider. * * *

    Issued by the Commission on May 2, 1997.
Margaret P. Crenshaw,
Secretary.
[FR Doc. 97-12191 Filed 5-8-97; 8:45 am]
BILLING CODE 7710-FW-P