mrjob.inline - debugger-friendly local testing¶
-
class
mrjob.inline.
InlineMRJobRunner
(mrjob_cls=None, **kwargs)¶ Runs an
MRJob
in the same process, so it’s easy to attach a debugger.This is the default way to run jobs (we assume you’ll spend some time debugging your job before you’re ready to run it on EMR or Hadoop).
Unlike other runners,
InlineMRJobRunner
‘srun()
method raises the actual exception that caused a step to fail (rather thanStepFailedException
).To more accurately simulate your environment prior to running on Hadoop/EMR, use
-r local
(seeLocalMRJobRunner
).New in version 0.6.8: can run
SparkStep
s via thepyspark
library.
-
InlineMRJobRunner.
__init__
(mrjob_cls=None, **kwargs)¶ InlineMRJobRunner
takes the same keyword args asMRJobRunner
. However, please note that hadoop_input_format, hadoop_output_format, and partitioner are ignored because they require Java. If you need to test these, consider starting up a standalone Hadoop instance and running your job with-r hadoop
.