Abstract

In engineering systems design, designers iteratively go back and forth between different design stages to explore the design space and search for the best design solution that satisfies all design constraints. For complex design problems, human has shown surprising capability in effectively reducing the dimensionality of design space and quickly converging it to a reasonable range for algorithms to step in and continue the search process. Therefore, modeling how human designers make decisions in such a sequential design process can help discover beneficial design patterns, strategies, and heuristics, which are essential to the development of new algorithms embedded with human intelligence to augment the computational design. In this paper, we develop a deep learning-based approach to model and predict designers’ sequential decisions in the systems design context. The core of this approach is an integration of the function-behavior-structure (FBS) model for design process characterization and the long short-term memory unit (LSTM) model for deep leaning. This approach is demonstrated in two case studies on solar energy system design, and its prediction accuracy is evaluated benchmarking on several commonly used models for sequential design decisions, such as the Markov Chain model, the Hidden Markov Chain model, and the random sequence generation model. The results indicate that the proposed approach outperforms the other traditional models. This implies that during a system design task, designers are very likely to rely on both short-term and long-term memory of past design decisions in guiding their future decision-making in the design process. Our approach can support human–computer interactions in design and is general to be applied in other design contexts as long as the sequential data of design actions are available.

References

1.
Rahman
,
M. H.
,
Gashler
,
M.
,
Xie
,
C.
, and
Sha
,
Z.
,
2018
, “
Automatic Clustering of Sequential Design Behaviors
,”
ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Quebec City, Quebec, Canada
,
Aug. 26–29
.
2.
Panchal
,
J. H.
,
Sha
,
Z.
, and
Kannan
,
K. N.
,
2017
, “
Understanding Design Decisions Under Competition Using Games With Information Acquisition and a Behavioral Experiment
,”
ASME J. Mech. Des.
,
139
(
9
), p.
091402
. 10.1115/1.4037253
3.
Sexton
,
T.
, and
Ren
,
M. Y.
,
2017
, “
Learning an Optimization Algorithm Through Human Design Iterations
,”
ASME J. Mech. Des.
,
139
(
10
), p.
101404
. 10.1115/1.4037344
4.
Baxter
,
G.
, and
Sommerville
,
I.
,
2011
, “
Socio-Technical Systems: From Design Methods to Systems Engineering
,”
Interact. Comput.
,
23
(
1
), pp.
4
17
. 10.1016/j.intcom.2010.07.003
5.
Sha
,
Z.
,
Chaudhari
,
A. M.
, and
Panchal
,
J. H.
,
2019
, “
Modeling Participation Behaviors in Design Crowdsourcing Using a Bipartite Network-Based Approach
,”
ASME J. Comput. Inf. Sci. Eng.
,
19
(
3
), p.
031010
. 10.1115/1.4042639
6.
Fuge
,
M.
,
Tee
,
K.
,
Agogino
,
A.
, and
Maton
,
N.
,
2014
, “
Analysis of Collaborative Design Networks: A Case Study of Openideo
,”
ASME J. Comput. Inf. Sci. Eng.
,
14
(
2
), p.
021009
. 10.1115/1.4026510
7.
Brockmann
,
E. N.
, and
Anthony
,
W. P.
,
1998
, “
The Influence of Tacit Knowledge and Collective Mind on Strategic Planning
,”
J. Manag. Issues
,
10
(
2
), pp.
204
222
.
8.
Collobert
,
R.
, and
Weston
,
J.
,
2008
, “
A Unified Architecture for Natural Language Processing: Deep Neural Networks With Multitask Learning
,”
Proceedings of the 25th International Conference on Machine Learning
,
Helsinki, Finland
,
July
, pp
160
167
.
9.
Miotto
,
R.
,
Wang
,
F.
,
Wang
,
S.
,
Jiang
,
X.
, and
Dudley
,
J. T.
,
2017
, “
Deep Learning for Healthcare: Review, Opportunities and Challenges
,”
Brief. Bioinform.
,
19
(
6
), pp.
1236
1246
. 10.1093/bib/bbx044
10.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2016
, “
Deep Residual Learning for Image Recognition
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Las Vegas, NV
,
June 27–30
, pp.
770
778
. http://dx.doi.org/10.1109/CVPR.2016.90
11.
Amershi
,
S.
,
Conati
,
C.
, and
Maclaren
,
H.
,
2006
, “
Using Feature Selection and Unsupervised Clustering to Identify Affective Expressions in Educational Games
,”
In Proceedings of The Intelligent Tutoring Systems Workshop on Motivational and Affective Issues in ITS (ITS 2006)
,
Jhongli, Taiwan
,
June 26–30
.
12.
Kan
,
J. W. T.
,
Gero
,
J. S.
, and
Press
,
C. R. C.
,
2009
, “
Using the FBS Ontology to Capture Semantic Design Information in Design Protocol Studies
,”
About: Designing. Analysing Design Meetings
, pp.
213
229
.
13.
Yu
,
R. O. N. G.
, and
Gero
,
J.S.
,
2015
, “
An Empirical Foundation for Design Patterns in Parametric Design
,”
20th International Conference of the Association for Computer-Aided Architectural Design Research in Asia (CAADRIA)
,
Daegu, South Korea
,
May
, pp.
20
23
.
14.
McComb
,
C.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2017
, “
Capturing Human Sequence-Learning Abilities in Configuration Design Tasks Through Markov Chains
,”
ASME J. Mech. Des.
,
139
(
9
), p.
091101
. 10.1115/1.4037185
15.
McComb
,
C.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2017
, “
Mining Process Heuristics From Designer Action Data via Hidden Markov Models
,”
ASME J. Mech. Des.
,
139
(
11
), p.
111412
. 10.1115/1.4037308
16.
Sha
,
Z.
,
Kannan
,
K. N.
, and
Panchal
,
J. H.
,
2015
, “
Behavioral Experimentation and Game Theory in Engineering Systems Design
,”
ASME J. Mech. Des.
,
137
(
5
), p.
051405
. 10.1115/1.4029767
17.
Chaudhari
,
A. M.
, and
Panchal
,
J. H.
,
2019
, “An Experimental Study of Human Decisions in Sequential Information Acquisition in Design: Impact of Cost and Task Complexity,”
Research Into Design for a Connected World
,
Springer
,
Singapore
, pp.
321
332
.
18.
Chaudhari
,
A. M.
,
Bilionis
,
I.
, and
Panchal
,
J. H.
,
2020
, “
Descriptive Models of Sequential Decisions in Engineering Design: An Experimental Study
,”
ASME J. Mech. Des.
,
142
(
8
), p.
081704
. 10.1115/1.4045605
19.
Shergadwala
,
M.
,
Bilionis
,
I.
,
Kannan
,
K. N.
, and
Panchal
,
J. H.
,
2018
, “
Quantifying the Impact of Domain Knowledge and Problem Framing on Sequential Decisions in Engineering Design
,”
ASME J. Mech. Des.
,
140
(
10
), p.
101402
. 10.1115/1.4040548
20.
Shergadwala
,
M. N.
,
Panchal
,
J. H.
, and
Bilionis
,
I.
,
2019
, “
Quantifying the Influence of Information Sharing About Competitor’s Performance on a Participant’s Sequential Design Behaviors in Design Contests
,”
ASME J. Mech. Des.
, under review.
21.
Smith
,
R. P.
, and
Eppinger
,
S. D.
,
1997
, “
A Predictive Model of Sequential Iteration in Engineering Design
,”
Manage. Sci.
,
43
(
8
), pp.
1104
1120
. 10.1287/mnsc.43.8.1104
22.
Miller
,
S. W.
,
Simpson
,
T. W.
, and
Yukish
,
M. A.
,
2015
, “
Design as a Sequential Decision Process: A Method for Reducing Design set Space Using Models to Bound Objectives
,”
ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Boston, MA
,
Aug. 2–5
, p.
V02AT03A020
.
23.
Griffin
,
S.
,
Welton
,
N. J.
, and
Claxton
,
K.
,
2010
, “
Exploring the Research Decision Space: The Expected Value of Information for Sequential Research Designs
,”
Med. Decis. Mak.
,
30
(
2
), pp.
155
162
. 10.1177/0272989X09344746
24.
Meier
,
C.
,
Yassine
,
A. A.
, and
Browning
,
T. R.
,
2007
, “
Design Process Sequencing With Competent Genetic Algorithms
,”
ASME J. Mech. Des.
,
129
(
6
), pp.
566
585
. 10.1115/1.2717224
25.
Duff
,
M. O.
, and
Barto
,
A.
,
2002
,
Optimal Learning: Computational Procedures for Bayes-Adaptive Markov Decision Processes
,
University of Massachusetts at Amherst
,
Amherst, MA
.
26.
Raina
,
A.
,
McComb
,
C.
, and
Cagan
,
J.
,
2019
, “
Learning to Design From Humans: Imitating Human Designers Through Deep Learning
,”
ASME J. Mech. Des.
,
141
(
11
), p.
111102
. 10.1115/detc2019-97399
27.
Oh
,
S.
,
Jung
,
Y.
,
Kim
,
S.
,
Lee
,
I.
, and
Kang
,
N.
,
2019
, “
Deep Generative Design: Integration of Topology Optimization and Generative Models
,”
ASME J. Mech. Des.
,
141
(
11
), p.
111405
. 10.1115/1.4044229
28.
Stump
,
G. M.
,
Miller
,
S. W.
,
Yukish
,
M. A.
,
Simpson
,
T. W.
, and
Tucker
,
C.
,
2019
, “
Spatial Grammar-Based Recurrent Neural Network for Design Form and Behavior Optimization
,”
ASME J. Mech. Des.
,
141
(
12
), p.
124501
. 10.1115/1.4044398
29.
Sundermeyer
,
M.
,
Ney
,
H.
, and
Schlüter
,
R.
,
2015
, “
From Feedforward to Recurrent LSTM Neural Networks for Language Modeling
,”
IEEE/ACM Trans. Audio, Speech, Lang. Process.
,
23
(
3
), pp.
517
529
. 10.1109/TASLP.2015.2400218
30.
Hanin
,
B.
,
2019
, “
Universal Function Approximation by Deep Neural Nets with Bounded Width and Relu Activations
,”
Mathematics
,
7
(
10
), pp.
992
. 10.3390/math7100992
31.
Bengio
,
Y.
,
Simard
,
P.
, and
Frasconi
,
P.
,
1994
, “
Learning Long-Term Dependencies With Gradient Descent is Difficult
,”
IEEE Trans. Neural Networks
,
5
(
2
), pp.
157
166
. 10.1109/72.279181
32.
Hochreiter
,
S.
, and
Schmidhuber
,
J.
,
1997
, “
Long Short-Term Memory
,”
Neural Comput.
,
9
(
8
), pp.
1735
1780
. 10.1162/neco.1997.9.8.1735
33.
Gero
,
J. S.
,
1990
, “
Design Prototypes: A Knowledge Representation Schema for Design
,”
AI Mag.
,
11
(
4
), p.
26
.
34.
Xie
,
C.
,
Schimpf
,
C.
,
Chao
,
J.
,
Nourian
,
S.
, and
Massicotte
,
J.
,
2018
, “
Learning and Teaching Engineering Design Through Modeling and Simulation on a CAD Platform
,”
Comput. Appl. Eng. Educ.
,
26
(
4
), pp.
824
840
. 10.1002/cae.21920
35.
Rahman
,
M. H.
,
Schimpf
,
C.
,
Xie
,
C.
, and
Sha
,
Z.
,
2019
, “
A Computer-Aided Design Based Research Platform for Design Thinking Studies
,”
ASME J. Mech. Des.
,
141
(
12
), p.
121102
. 10.1115/1.4044395
36.
Verleysen
,
M.
, and
François
,
D.
,
2005
, “
The Curse of Dimensionality in Data Mining and Time Series Prediction
,”
International Work-Conference on Artificial Neural Networks
,
Barcelona, Spain
,
June
, pp
758
770
.
37.
Potdar
,
K.
,
Pardawala
,
T. S.
, and
Pai
,
C. D.
,
2017
, “
A Comparative Study of Categorical Variable Encoding Techniques for Neural Network Classifiers
,”
Int. J. Comput. Appl.
,
175
(
4
), pp.
7
9
.
38.
Kohavi
,
R.
, et al
,
1995
, “
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
,”
Ijcai
,
14
(
2
), pp.
1137
1145
.
39.
Chollet
,
F.
,
2017
,
Deep Learning with Python
, 1st,
Manning Publications Co.
,
USA
.
40.
Friedman
,
J.
,
Hastie
,
T.
, and
Tibshirani
,
R.
,
2001
,
The Elements of Statistical Learning
, Vol.
1
, No.
10
,
Springer Series in Statistics
,
New York
.
41.
Ruder
,
S.
,
2016
, “
An Overview of Gradient Descent Optimization Algorithms
,”
arXiv Prepr. arXiv1609.04747
.
42.
Lee
,
K. C.
,
Phon-Amnuaisuk
,
S.
, and
Ting
,
C. Y.
,
2010
, “
A Comparison of HMM, Naive Bayesian, and Markov Model in Exploiting Knowledge Content in Digital Ink: A Case Study on Handwritten Music Notation Recognition
,”
2010 IEEE International Conference on Multimedia and Expo
,
Singapore
,
July 19–23
, pp.
292
297
. http://dx,doi.org/10.1109/ICME.2010.5583292
43.
Sha
,
Z.
,
Huang
,
Y
,
Fu
,
S
,
Wang
,
M
,
Fu
,
Y
,
Contractor
,
N
, and
Chen
,
W
,
2018
, “
A Network-Based Approach to Modeling and Predicting Product Consideration Relations
,”
Complexity
,
2018
, pp.
1
14
. 10.1155/2018/2753638
44.
Fawcett
,
T.
,
2006
, “
An Introduction to ROC Analysis
,”
Pattern Recognit. Lett.
,
27
(
8
), pp.
861
874
. 10.1016/j.patrec.2005.10.010
45.
Srivastava
,
N.
,
Hinton
,
G.
,
Krizhevsky
,
A.
,
Sutskever
,
I.
, and
Salakhutdinov
,
R.
,
2014
, “
Dropout: A Simple Way to Prevent Neural Networks From Overfitting
,”
J. Mach. Learn. Res.
,
15
(
1
), pp.
1929
1958
.
46.
Mikolov
,
T.
,
Sutskever
,
I.
,
Chen
,
K.
,
Corrado
,
G. S.
, and
Dean
,
J.
,
2013
, “
Distributed Representations of Words and Phrases and Their Compositionality
,”
Advances in Neural Information Processing Systems
,
Lake Tahoe, NV
,
December
, pp
3111
3119
.
47.
Pennington
,
J.
,
Socher
,
R.
, and
Manning
,
C. D.
,
2014
, “
Glove: Global Vectors for Word Representation
,”
Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)
,
Doha, Qatar
,
October
, pp.
1532
1543
.
You do not currently have access to this content.