compat-Testing.html 8.9 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199
  1. <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
  2. <html>
  3. <!-- Copyright (C) 1988-2017 Free Software Foundation, Inc.
  4. Permission is granted to copy, distribute and/or modify this document
  5. under the terms of the GNU Free Documentation License, Version 1.3 or
  6. any later version published by the Free Software Foundation; with the
  7. Invariant Sections being "Funding Free Software", the Front-Cover
  8. Texts being (a) (see below), and with the Back-Cover Texts being (b)
  9. (see below). A copy of the license is included in the section entitled
  10. "GNU Free Documentation License".
  11. (a) The FSF's Front-Cover Text is:
  12. A GNU Manual
  13. (b) The FSF's Back-Cover Text is:
  14. You have freedom to copy and modify this GNU Manual, like GNU
  15. software. Copies published by the Free Software Foundation raise
  16. funds for GNU development. -->
  17. <!-- Created by GNU Texinfo 5.2, http://www.gnu.org/software/texinfo/ -->
  18. <head>
  19. <title>GNU Compiler Collection (GCC) Internals: compat Testing</title>
  20. <meta name="description" content="GNU Compiler Collection (GCC) Internals: compat Testing">
  21. <meta name="keywords" content="GNU Compiler Collection (GCC) Internals: compat Testing">
  22. <meta name="resource-type" content="document">
  23. <meta name="distribution" content="global">
  24. <meta name="Generator" content="makeinfo">
  25. <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  26. <link href="index.html#Top" rel="start" title="Top">
  27. <link href="Option-Index.html#Option-Index" rel="index" title="Option Index">
  28. <link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
  29. <link href="Testsuites.html#Testsuites" rel="up" title="Testsuites">
  30. <link href="Torture-Tests.html#Torture-Tests" rel="next" title="Torture Tests">
  31. <link href="profopt-Testing.html#profopt-Testing" rel="prev" title="profopt Testing">
  32. <style type="text/css">
  33. <!--
  34. a.summary-letter {text-decoration: none}
  35. blockquote.smallquotation {font-size: smaller}
  36. div.display {margin-left: 3.2em}
  37. div.example {margin-left: 3.2em}
  38. div.indentedblock {margin-left: 3.2em}
  39. div.lisp {margin-left: 3.2em}
  40. div.smalldisplay {margin-left: 3.2em}
  41. div.smallexample {margin-left: 3.2em}
  42. div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
  43. div.smalllisp {margin-left: 3.2em}
  44. kbd {font-style:oblique}
  45. pre.display {font-family: inherit}
  46. pre.format {font-family: inherit}
  47. pre.menu-comment {font-family: serif}
  48. pre.menu-preformatted {font-family: serif}
  49. pre.smalldisplay {font-family: inherit; font-size: smaller}
  50. pre.smallexample {font-size: smaller}
  51. pre.smallformat {font-family: inherit; font-size: smaller}
  52. pre.smalllisp {font-size: smaller}
  53. span.nocodebreak {white-space:nowrap}
  54. span.nolinebreak {white-space:nowrap}
  55. span.roman {font-family:serif; font-weight:normal}
  56. span.sansserif {font-family:sans-serif; font-weight:normal}
  57. ul.no-bullet {list-style: none}
  58. -->
  59. </style>
  60. </head>
  61. <body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
  62. <a name="compat-Testing"></a>
  63. <div class="header">
  64. <p>
  65. Next: <a href="Torture-Tests.html#Torture-Tests" accesskey="n" rel="next">Torture Tests</a>, Previous: <a href="profopt-Testing.html#profopt-Testing" accesskey="p" rel="prev">profopt Testing</a>, Up: <a href="Testsuites.html#Testsuites" accesskey="u" rel="up">Testsuites</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Option-Index.html#Option-Index" title="Index" rel="index">Index</a>]</p>
  66. </div>
  67. <hr>
  68. <a name="Support-for-testing-binary-compatibility"></a>
  69. <h3 class="section">7.8 Support for testing binary compatibility</h3>
  70. <p>The file <samp>compat.exp</samp> provides language-independent support for
  71. binary compatibility testing. It supports testing interoperability of
  72. two compilers that follow the same ABI, or of multiple sets of
  73. compiler options that should not affect binary compatibility. It is
  74. intended to be used for testsuites that complement ABI testsuites.
  75. </p>
  76. <p>A test supported by this framework has three parts, each in a
  77. separate source file: a main program and two pieces that interact
  78. with each other to split up the functionality being tested.
  79. </p>
  80. <dl compact="compact">
  81. <dt><samp><var>testname</var>_main.<var>suffix</var></samp></dt>
  82. <dd><p>Contains the main program, which calls a function in file
  83. <samp><var>testname</var>_x.<var>suffix</var></samp>.
  84. </p>
  85. </dd>
  86. <dt><samp><var>testname</var>_x.<var>suffix</var></samp></dt>
  87. <dd><p>Contains at least one call to a function in
  88. <samp><var>testname</var>_y.<var>suffix</var></samp>.
  89. </p>
  90. </dd>
  91. <dt><samp><var>testname</var>_y.<var>suffix</var></samp></dt>
  92. <dd><p>Shares data with, or gets arguments from,
  93. <samp><var>testname</var>_x.<var>suffix</var></samp>.
  94. </p></dd>
  95. </dl>
  96. <p>Within each test, the main program and one functional piece are
  97. compiled by the GCC under test. The other piece can be compiled by
  98. an alternate compiler. If no alternate compiler is specified,
  99. then all three source files are all compiled by the GCC under test.
  100. You can specify pairs of sets of compiler options. The first element
  101. of such a pair specifies options used with the GCC under test, and the
  102. second element of the pair specifies options used with the alternate
  103. compiler. Each test is compiled with each pair of options.
  104. </p>
  105. <p><samp>compat.exp</samp> defines default pairs of compiler options.
  106. These can be overridden by defining the environment variable
  107. <code>COMPAT_OPTIONS</code> as:
  108. </p>
  109. <div class="smallexample">
  110. <pre class="smallexample">COMPAT_OPTIONS=&quot;[list [list {<var>tst1</var>} {<var>alt1</var>}]
  111. &hellip;[list {<var>tstn</var>} {<var>altn</var>}]]&quot;
  112. </pre></div>
  113. <p>where <var>tsti</var> and <var>alti</var> are lists of options, with <var>tsti</var>
  114. used by the compiler under test and <var>alti</var> used by the alternate
  115. compiler. For example, with
  116. <code>[list [list {-g -O0} {-O3}] [list {-fpic} {-fPIC -O2}]]</code>,
  117. the test is first built with <samp>-g -O0</samp> by the compiler under
  118. test and with <samp>-O3</samp> by the alternate compiler. The test is
  119. built a second time using <samp>-fpic</samp> by the compiler under test
  120. and <samp>-fPIC -O2</samp> by the alternate compiler.
  121. </p>
  122. <p>An alternate compiler is specified by defining an environment
  123. variable to be the full pathname of an installed compiler; for C
  124. define <code>ALT_CC_UNDER_TEST</code>, and for C++ define
  125. <code>ALT_CXX_UNDER_TEST</code>. These will be written to the
  126. <samp>site.exp</samp> file used by DejaGnu. The default is to build each
  127. test with the compiler under test using the first of each pair of
  128. compiler options from <code>COMPAT_OPTIONS</code>. When
  129. <code>ALT_CC_UNDER_TEST</code> or
  130. <code>ALT_CXX_UNDER_TEST</code> is <code>same</code>, each test is built using
  131. the compiler under test but with combinations of the options from
  132. <code>COMPAT_OPTIONS</code>.
  133. </p>
  134. <p>To run only the C++ compatibility suite using the compiler under test
  135. and another version of GCC using specific compiler options, do the
  136. following from <samp><var>objdir</var>/gcc</samp>:
  137. </p>
  138. <div class="smallexample">
  139. <pre class="smallexample">rm site.exp
  140. make -k \
  141. ALT_CXX_UNDER_TEST=${alt_prefix}/bin/g++ \
  142. COMPAT_OPTIONS=&quot;<var>lists as shown above</var>&quot; \
  143. check-c++ \
  144. RUNTESTFLAGS=&quot;compat.exp&quot;
  145. </pre></div>
  146. <p>A test that fails when the source files are compiled with different
  147. compilers, but passes when the files are compiled with the same
  148. compiler, demonstrates incompatibility of the generated code or
  149. runtime support. A test that fails for the alternate compiler but
  150. passes for the compiler under test probably tests for a bug that was
  151. fixed in the compiler under test but is present in the alternate
  152. compiler.
  153. </p>
  154. <p>The binary compatibility tests support a small number of test framework
  155. commands that appear within comments in a test file.
  156. </p>
  157. <dl compact="compact">
  158. <dt><code>dg-require-*</code></dt>
  159. <dd><p>These commands can be used in <samp><var>testname</var>_main.<var>suffix</var></samp>
  160. to skip the test if specific support is not available on the target.
  161. </p>
  162. </dd>
  163. <dt><code>dg-options</code></dt>
  164. <dd><p>The specified options are used for compiling this particular source
  165. file, appended to the options from <code>COMPAT_OPTIONS</code>. When this
  166. command appears in <samp><var>testname</var>_main.<var>suffix</var></samp> the options
  167. are also used to link the test program.
  168. </p>
  169. </dd>
  170. <dt><code>dg-xfail-if</code></dt>
  171. <dd><p>This command can be used in a secondary source file to specify that
  172. compilation is expected to fail for particular options on particular
  173. targets.
  174. </p></dd>
  175. </dl>
  176. <hr>
  177. <div class="header">
  178. <p>
  179. Next: <a href="Torture-Tests.html#Torture-Tests" accesskey="n" rel="next">Torture Tests</a>, Previous: <a href="profopt-Testing.html#profopt-Testing" accesskey="p" rel="prev">profopt Testing</a>, Up: <a href="Testsuites.html#Testsuites" accesskey="u" rel="up">Testsuites</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Option-Index.html#Option-Index" title="Index" rel="index">Index</a>]</p>
  180. </div>
  181. </body>
  182. </html>