Scalable Parallel Finite Element Computations of Rayleigh-Benard-Marangoni Problems in a Microgravity Environment

Project Director: Graham F. Carey

NASA ESS Cooperative Agreement Number: NCCS5-154

PERIOD COVERED BY THE REPORT: August 16, 1997-August 15, 1998

Objectives:

This research project concerns the analysis and simulation of coupled viscous flow and transport processes including free surface thermocapillary effects. Of particular interest are thermocapillary surface tension effects in microgravity flow and transport phenomena and applications for manufacturing, life support processes and accident analysis relevant to the U.S. space station, space shuttle program and future space projects. The main thrust of the present research is directed to the development of a highly parallel distributed simulation capability for analyzing 3-D transient and steady state coupled viscous flow and transport processes in this class. This implies an emphasis on parallel algorithm research and methodology for efficient scalable high performance simulations that can be applied to microgravity problems of fundamental interest. We emphasize that the simulator will also be applicable to other coupled flow and transport problems of interest to NASA, U.S. manufacturing, and Science and Engineering. e.g. it can be applied to terrestrial manufacturing processes and will be particularly important to problems involving thin films and coatings since surface tension is also dominant (over gravity) in these applications. Part of the research work also concerns supporting experimental studies on thermocapillary effects in thin films.

Approach:

We develop a system of governing partial differential equations for coupled viscous incompressible flow and heat transfer that include both buoyancy and thermocapillary surface tension effects. The numerical approach is based on a Galerkin finite element formulation using hexahedral elements with piecewise continuous triquadratic approximation for the velocity, temperature (or species) fields and trilinear pressure finite element basis. Parallelism is achieved using a domain decomposition of the flow domain and mesh. The resulting discretized problem leads to a sequence of sparse algebraic systems that are solved using a parallel form of generalized conjugate gradient solver. Special attention has been paid to parallel performance aspects to meet the performance milestones specified in the contract. This implies that such issues as efficient parallel communication and overlapping communication and computation are addressed as well as use of BLAS for computational kernels. The experimental work has been directed to fundamental studies of free surface phenomena for thin films and is being carried out in the Experimental Flows Lab of the Center for Nonlinear Dynamics.

Accomplishments

The performance milestones and schedule for the previous year and the performance levels achieved are summarized below:

Milestones

6. Achieve 50 GigaFlops sustained performance using on vendor testbed. (25 GigaFlops on code suite for test problem with scaling analysis). Submission to National HPCC Software Exchange of well documented algorithms, subroutines and tools developed in building microgravity analysis and primary algorithms.

7. Parallel Algorithm Research. Visit to NASA. Complexity analysis and scalability studies on RBM and liquid bridge microgravity problem. Comparison studies with flow experiments.

8. 100 GigaFlop kernel performance achieved. RBM and Bridgeman application studies. Submitted FY 98 annual report to sponsor via WWW.

The major scheduled milestones for this year have been met or exceeded. In addition, progress has been made on experimental studies related to observations and analysis concerning thin film and liquid bridge phenomena. Further details are provided in the Attachments.

Status and Plans

As indicated above, significant progress has been made on the simulator development and the major milestone goals are being met or exceeded on schedule. During the first part of the year the formulation was extended to include a fully coupled treatment as well as the previous decoupled algorithm. Both are options that can be selected by the analyst and both have been benchmarked on the T3E. Both the 50 gigaflop and 100 gigaflop performance targets have been met and verified at NASA Goddard.

Presentations and Publications

A number of publications and presentations on the underlying methodology and related work on algorithms were presented or are scheduled for presentation this year. e.g. Some aspects of the flow formulation, finite element discretization, solution algorithm, parallel aspects and scalability studies were presented at Supercomputing '97. Reprints of the paper are on the website. Two papers related to the experimental component of the project and dealing with the long-wavelength instabilities and instability threshold in the Rayleigh-Benard-Maragoni problem have been published (See Attachment III for more particulars).

Journal Publications

Barragy, E. and G. F. Carey,  Stream Function-Vorticity Driven Cavity
Solution using p Finite Elements, Computers and Fluids, 26, 5, 453-468,
1997.

Carey, G. F. and A. I. Pehlivanov, Local Error Estimation and Adaptive
Remeshing Scheme for Least-Squares Mixed Finite Elements, Computer Meth.
Appld. Mech. and Eng., 150, 125-131, 1997.

Carey, G. F.,  A. Pehlivanov, Y. Shen, A. Bose, and K. C. Wang, Least
Squares Finite Elements for Fluid Flow and Transport, Int'. J. Num. Meths.
in Fluids, Vol 27, 97-107, 1998.

Lorber, A.A., G. F. Carey, S. W. Bova and C. Harle, Accelerated Solution of
Non-Linear Flow Problems using Chebyshev Iteration Polynomial-Based
Runge-Kutta Recursions, Computers and Fluids, Vol 27, No. 7, 769-782, 1998.

Petrov, Valery, A. Haaning, K. A. Muehlner, S. J. Van Hook and H.L.
Swinney, Model-independent nonlinear control algorithm with application to
a liquid bridge experiment, Physical Review E, Vol. 58, No. 1, 427-433,
1998.

Swinney, H. L.,  Emergence and Evolution of Patterns, From Critical
Problems in Physics, Eds. V. L. Fitch, D. R. Marlow and M.A.E. Dementi,
Princeton University Press, pp. 51-74, 1997.

Van Hook, Stephen J., M. F. Schatz, J. B. Swift, W. D. McCormick and H. L.
Swinney,  "Long-wavelength surface-tension-driven Benard convection:
experiment and theory, "Journal of Fluid Mechanics,  345, 45-78, 1997.

Van Hook, Stephen J. and J. B. Swift, "Comment II on 'Instability Threshold
in the Benard-Maragoni problem'," Physical Review E, 56, 4897-4899, 1997.

Bose, A. and G. F. Carey, A Class of Data Structures and Object-Oriented
Implementation for Finite Element Methods on Distributed Memory Systems,
Computer Meth. Appld. Mech. and Eng, In Press, June 1998.

Carey, G. F., R. McLay, G. Bicken, B. Barth, Parallel Finite Element
Solution of 3D Rayleigh-Benard-Marangoni Flows, Int'. J. Num. Meths. in
Fluids, Submitted July 1998.

Becerril, R., S. J. Van Hook and J. B. Swift, "The influence of interface
profile on the onset of long-wavelength Marangoni convection,"  Submitted
to Phys. of Fluids, 1998.

Proceedings Publications*

Plaza, A., G. F. Carey and Miguel Padron, A 3D Refinement/Derefinement
Algorithm for Solving Evolution Problems, Proceedings IMACS World Congress,
Aug 24-29, 1997, Berlin, Germany.

Carey, G. F., Parallel CFD, Proceedings of CTAC '97, The University of
Adelaide, Australia, September 29-October 1, 1997.

Carey, G.F., C. Harle, R. McLay and S. Swift, MPP Solution of
Rayleigh-Benard-Marangoni Flows, Proceedings of Supercomputing '97,
November 1997, San Jose, CA. [postscript version]

Carey, G., A. Bose and R. McLay, Some Aspects of Distributed Parallel
Viscous Flow and Transport Computation, 10th Int'l Conf on Finite Elements
in Fluids, Tucson, AZ, Jan 5-8, 1998.

Valli, A.M.P. , G.F. Carey, and A.L.G.A. Coutinho, Finite Element
Simulation and Control of Nonlinear Flow and Reactive Transport, 10th Int'l
Conf on Finite Elements in Fluids, Tucson, AZ, Jan 5-8, 1998.

Bose, A. and G. F. Carey, Least-Squares Finite Element Methods for
Incompressible non-Newtonian Flows,  10th Int'l Conf on Finite Elements in
Fluids, Tucson, AZ, Jan 5-8, 1998.

Bose, A. and G. F. Carey, A Parallel Multiple Front Algorithm for p
Adaptive Finite Element Solution of Viscous Incompressible Flows, High
Performance Computing '98 Proceedings, Boston, MA, April 1998.

Carey, G. F., R. McLay, C. Harle, S. Swift and W. Spotz, Large-Scale
Parallel Distributed Solution of 3-D Microgravity Flows, High Performance
Computing '98 Proceedings, Boston, MA, April 1998.

Carey, G. F., A. I. Pehlivanov, R. C. Mastroleo, R. McLay, Y. Shen, V.
Carey and R. Dutton, Hierarchic Visualization and Parallel Computation,
High Performance Computing '98 Proceedings, Boston, MA, April 1998.

Carey, G. F., The Finite Element Method: Its Application in Engineering
Mathematics, Proceedings of EMAC'98, Adelaide, Australia, July 1998.

Presentations

*The above Proceedings papers were also presented at the associated
technical meetings.  In addition, the following presentations were made.

Carey, G. F., Parallel Finite Element Solution of Coupled Transport
Problems, FE Rodeo, A&M University, March 6, 1998.

Carey, G. F., G. Bicken and V. Carey, Intergrid Projections Involving
Constraints, SIAM Annual Meeting, Toronto, CANADA, July 13, 1998.

Carey, G. F., Presentation at NASA Semi-Annual Program Review Meeting,
Washington, DC, April 1998.

Carey, G. F., AMR Developments and Current Trends, AMR Workshop, NASA
Goddard, Greenbelt, MD, April 3-4, 1998.

Poster Display, Seventh Annual Industrial Affiliates Meeting,  Center for
Subsurface Modeling, The University of Texas at Austin,November 5-6, 1997.

Carey, G. F., Summary presentation at NASA Reveiw Meeting, San Jose, CA,
November 1997.

Symposia/Dissemination

A special workshop on Adaptive Mesh Refinement was organized and coordinated by Dr. Carey jointly with Steve Zalesak, Tom Cwik, Andrea Malagoli and Ken Powell at Goddard. This workshop was well attended and very successful.

Collaboration/NASA Visits

During the year Dr. Carey made visits to NASA to discuss progress on the research, and coordinate/plan research activities. These work trips included discussions on the main parallel algorithm work and applications studies as well as problems in visualizing complex 3D flows. S. Swift, the Cray scientist at Goddard, has been collaborating closely in the research and has consulted regularly and worked with us on performance studies. He has also visited the University to facilitate the interaction. Dr. Spotz from NCAR spent two weeks in Fall semester collaborating on the project. To facilitate close interaction, a post-doctoral associate, Dr. B. Davis, was in residence at NASA Goddard during part of the year while supported by the U.T. project budget. Remarks: We are very pleased with the progress on the research and parallel simulator, performance and scalability. We have also helped uncover hardware and compiler or other software bugs so this should be of general benefit to NASA and Cray application analysts. We have had to 'weather' some of these difficulties in due course and this has naturally incurred more time and effort than would be the case in a more stable computing environment.

List of Attachments

        I.     Experimental Studies
        II.    Software Submission Report, for 50 gigaflop milestone including current software version
        III.   Current Software Document, [postscript file]

CFDLab,
The University of Texas at Austin.
Please send the remarks to:
CFDLAB Web Team,
Last Updated:
Sep. 15, 1998.