viXard	People, today is the last one on talks during InfoSec
viXard	3 talks will complete this weeks sessions and we will do it with great talks
viXard	first, Jose Nazario will give us not one bt two straight talks :-) then, KF from USA, will close today´s and this year´s sessions of InfoSec
viXard	Jose :-)
jose_n	:)
jose_n	thank you vixard :)
jose_n	it's a pleasure and an honor to be here
jose_n	i was invited to join by sarnold a few months ago, and i am quite honored to be here among some really esteemed people.
jose_n	thanks for showing up on a weekend (morning for some of us in the USA)
jose_n	the talk here will be in english, and i think #redes will be the spanish translation
jose_n	i have the presentation translated on my website.
jose_n	http://www.monkey.org/~jose/presentations/czech-rubicon02.d/czech.html
jose_n	into french and spanish (via babelfish)
jose_n	and slides, too. if you can, follow along
jose_n	ok, let us begin! :)
jose_n	Lexical analysis in source code scanning
jose_n	(i will be pasting from the prepared talk)
jose_n	Note: this is a rough writeout of what will be presented at
jose_n	Uninet Infosec Infosec 2002 on Saturday, 20 April, 2002. A
jose_n	babelfish translation into spanish and french appears below. The
jose_n	slides are available on my website at:
jose_n	http://www.monkey.org/~jose/presentations/czech-rubicon02.d/
jose_n	... this page is available at
jose_n	http://www.monkey.org/~jose/presentations/czech-rubicon02.d/czech.html
jose_n	INTRODUCTION
jose_n	[slide 1]
jose_n	This talk is designed to introduce the application of lexical
jose_n	analysis to source code analysis. Specically, the design and
jose_n	implentation of the static analysis tool for the C programming
jose_n	language, named "czech", will be presented. Several inherent
jose_n	flaws in static analysis, the use of lexical analysis in source
jose_n	code reviews, and specific lessons learned from the failure in
jose_n	the implentation of czech will be discussed.
jose_n	[slide 2]
jose_n	Briefly, I will give an overview of some of the current research I
jose_n	am familiar with in this arena. Several people are attempting to
jose_n	leverage many, many years of source code evaluation methods
jose_n	and tools to automate security analysis of their code. This is an
jose_n	area of high interest for corporate and academic researchers
jose_n	alike. I will introduce what lexical code generation tools are and
jose_n	how they operate, specically the use of the "flex" tool, a freely
jose_n	available lex parser. I will then outline typical strategies for
jose_n	these tools, and then delve into the specific challenges faced in
jose_n	the execution of the czech tool, both for the tool and the author
jose_n	as well as for users of the tool.
jose_n	[for those of you just joining us, slides and material at
jose_n	http://www.monkey.org/~jose/presentations/czech-rubicon02.d/
jose_n	]
jose_n	[slide 3]
jose_n	Quickly, an overview of the presentation specifically about
jose_n	czech. I will discuss the design philosophy of czech, and then
jose_n	how I designed and then executed the implementation. The
jose_n	performance of czech is something I'm quite proud of,
jose_n	interestingly enough. I'll give some example of how it operates
jose_n	and show you the rough output of the tool, and discuss some of
jose_n	the outstanding bugs as it currently stands. Lastly, I will
jose_n	discuss some of the future directions for czech and the whole of
jose_n	project pedantic.
jose_n
jose_n	[slide 4]
jose_n	Despite all of the interest its getting these days, source code
jose_n	analysis is not anything new. Software engineering researchers
jose_n	have been doing this for years. In the security realm, this has
jose_n	become a hot area for researchers in recent years, and many big
jose_n	groups have chimed in. These include David Wagner (Univ California Berkeley), David
jose_n	Evans, (Univ Va)) and Microsoft.
jose_n
jose_n	The two big approaches are static analysis, looking at source
jose_n	code typically. Dynamic analysis, in contrast, takes the whole
jose_n	or parts of the program and runs it with varying input,
jose_n	monitoring the behavior of the program. Static analysis focuses
jose_n	on things like logical analysis of the design, flow analysis as it
jose_n	handles data, and the analysis of type qualifiers in arguments.
jose_n	All of these have their varying strengths, including the
jose_n	extendability, ease of use, or completeness of coverage.
jose_n
jose_n	[for those of you just tuning in, slides and material can be found at
jose_n	http://www.monkey.org/~jose/presentations/czech-rubicon02.d/ ]
jose_n	It should definitely be noted that no method gives total
jose_n	coverage, and each misses some aspects. Also, there is the
jose_n	Turing 'halting problem', where you can only examine a small
jose_n	fraction of the possible states of the code in any given time
jose_n	frame. As such, you have to focus in on some aspects and make
jose_n	tradeoffs between completeness and efficiency.
jose_n
jose_n	[slide 5]
jose_n	Project pedantic certainly owes a lot to some of the more
jose_n	accessible static analysis tools. Pscan was one of the first
jose_n	format string analysis tools, and could be used to find some
jose_n	common format string problems (see also KF's talk later today
jose_n	for what those mean). ITS4, and later RATS, were also some of
jose_n	the most commonly cited tools for common programming errors.
jose_n	Both are also based on lexical analysis, too. Flawfinder, based
jose_n	on python, works similarily to RATS, but makes some smarter
jose_n	decisions. Lastly, two great tools to note are Lclint and splint,
jose_n	both from the research of David Evans and coworkers in
jose_n	Virginia (USA).
jose_n
jose_n	Czech takes a lot of inspiration from these tools. In fact, I
jose_n	started writing it after a piece I wrote for Linux Journal on
jose_n	RATS, ITS4 and Flawfinder and getting rather upset with them.
jose_n	Also, during the design of czech, I spoke to David Wheeler, the
jose_n	author of Flawfinder. I tried to leverage a lot from this 'prior art'.
jose_n	I should also note, for humor's sake, that the test suite for czech
jose_n	breaks RATS and Flawfinder.
jose_n
jose_n	[slide 6]
jose_n	Lex is a tool for parsing input streams. A lexical analyzer looks
jose_n	at the patterns as they flow by and performs actions based on
jose_n	them. The lex tool takes the parser and generates C code based
jose_n	on that, which is then compiled. These patterns are based on
jose_n	regular expressions, which makes them powerful and easy to
jose_n	construct. The actions are based largely on C, with some
jose_n	macros and functions from the lex and yacc libraries.
jose_n
jose_n	[slide 7]
jose_n	Here we see an example of some of the lex code in the main
jose_n	scanner in czech. Some of the actions match some obvious
jose_n	things, like tabs, newlines, and single characters, or words. The
jose_n	function "unput()" is a lex function which places what you tell it
jose_n	to back on the stream. REJECT is a macro that unputs what you
jose_n	matched. Once you have started your action, you can continue
jose_n	to access the stream via the input() function.
jose_n
jose_n	This piece of code is pretty ugly, by the way. I really need to
jose_n	clean it up to make it more readable and do some proper
jose_n	grammar checking. Niels Provos, a friend of mine, read it and
jose_n	said, "It makes my brain hurt."
jose_n
jose_n	[slide 8]
jose_n	Like I mentioned above, you run this parser through the lex
jose_n	generator to generate C code. Then you can compile it:
jose_n
jose_n	$ lex -t file.l > file.c
jose_n	$ cc -c file.c
jose_n	$ cc -o file file.o -ll -ly
jose_n
jose_n	That runs the
jose_n	parser "file.l" through lex to generate some C code, which you
jose_n	then compile. In the final linking step you link in the lex and yacc
jose_n	libraries for specific functions from them. Pretty simple. The flex
jose_n	manpage is a great tutorial on lex, by the way.
jose_n	(thank you vern paxson!)
jose_n
jose_n	[for those of you just coming in, you can find slides and other material on my website at
jose_n	http://www.monkey.org/~jose/presentations/czech-rubicon02.d/ ]
jose_n
jose_n	[ also, if you would like to ask a question, please feel free to do so in #qc.]
jose_n
jose_n	[page 9]
jose_n	The project strategy was pretty simple, then, in the abstract,
jose_n	and would focus the efforts on a lex parser. A yacc parser is for
jose_n	later (more below).
jose_n
jose_n	To choice between a full C parser and a keyword tag was
jose_n	actually a decision between a complex design and a simple
jose_n	design. I chose the simpler one, but I will eventually have to
jose_n	move closer towards a C parser. Czech understands the
jose_n	improper usage of dangerous functions, like the printf() family
jose_n	for format string problems and the use of strcat() in loops,
jose_n	which should never be done.
jose_n
jose_n	[page 10]
jose_n	The design philosophy of czech was pretty important for me.
jose_n	The main points are:
jose_n
jose_n	. dont ask much of the developer
jose_n	its time consuming, error prone, and a big turnoff. an
jose_n	example is Lclint, a powwerful tool which requires
jose_n	annotations of the code for real use. Cqual is another
jose_n	example, which requires developers to recast their code
jose_n	into a logic structure. Instead, I was shooting for a drop
jose_n	in replacement for CC in your Makefile.
jose_n	. make the output easy to read
jose_n	don't give them too much more to worry about
jose_n	. make it extendable
jose_n	if you extend an API, you should be able to train the tool
jose_n	pretty easily. (i failed here miserably)
jose_n	. work pretty fast
jose_n	even though you will have to spend a lot more time
jose_n	looking over the output, you should still return in a
jose_n	reasonable amount of time
jose_n	. attempt to understand program flow
jose_n	czech divides it into ingress and egress (inbound and
jose_n	outbound) data flows
jose_n
jose_n	[for those of you who are new, you can find the materials for this talk, including slides and text in english, spanish and french on my website at
jose_n	http://www.monkey.org/~jose/presentations/czech-rubicon02.d/ ]
jose_n
jose_n	[page 11]
jose_n	The challenges facing any of these tools are very elemental.
jose_n	First you have false positives, and then you have false
jose_n	negatives. Each will lead to a decreased confidence in your tool,
jose_n	and discourage developers. You should be able to understand
jose_n	new functions, such as glib or BIND, making it useful for really
jose_n	large projects like GNOME. Lastly, remember than C is a very
jose_n	flexible language, and no one codes similarily. Your tools should
jose_n	be able to deal with this reality.
jose_n
jose_n	[page 12]
jose_n	Czech as it stands is an initial attempt to implement these
jose_n	ideas. It uses preprocessing in a first pass to expand macros
jose_n	and grab function prototypes, as well as variable declarations.
jose_n	Basic type qualifying is performed, as it builds two lists of data
jose_n	types. Data that is input from the user is going to be treated far
jose_n	more strictly than data that is stically defined. This is in the
jose_n	case of a string format issue, for example. Consider the
jose_n	improper handling of a string below:
jose_n
jose_n	static char *stuff = "this is a string";
jose_n	/* things happen */
jose_n	syslog (LOG_INFO, stuff);
jose_n
jose_n	Syslog has to worry about string formatting, but if you've defined a
jose_n	string yourself, you know what kind of data you have there, so
jose_n	you shouldn't throw a red flag on that (probably just a orange
jose_n	flag). Now, if you used a function like that and *stuff had user
jose_n	supplied content, then you should definitely check its formatting
jose_n	and its bounds. Czech attempts to do this for you with these split
jose_n	data types.
jose_n
jose_n	[page 13]
jose_n	Czech is the first available component of project pedantic,
jose_n	which is designed to be a set of source code analysis tools. Its
jose_n	written in about 1000 lines of code, and runs realy fast (see
jose_n	performance, below) and is pretty easy to use. Currently its not
jose_n	fully functional. While it can build the lists, they're not yet used.
jose_n	This is more a matter of time and motivation right now, but it
jose_n	should be pretty easy (using the OpenBSD QUEUE(3)
jose_n	interface). Lastly, there are several really show stopper bugs in
jose_n	it right now.
jose_n
jose_n	However, its actually ready to drop into place of your C compiler
jose_n	in many Makefiles. Also, I've used it to actually find some bugs
jose_n	and have reported them to the right people.
jose_n
jose_n	[if you are just joining, you can find slides and text on my website at
jose_n	http://www.monkey.org/~jose/presentations/czech-rubicon02.d/ ]
jose_n
jose_n	[if you have any questions, i would be happy to answer then now or after the presentation. please ask in #qc]
jose_n
jose_n	[page 14]
jose_n	Czechs key features are sumarized here:
jose_n
jose_n	. it knows a bit about safe usage
jose_n	a strcpy() with a constant source string is safer than a
jose_n	variable length string, but still not safe. And it can
jose_n	safely ignore the use of va_args() in the printf() family.
jose_n	. it knows a bit about unsafe usage
jose_n	the biggest one is the use of strcat() in a loop, which
jose_n	can be very dangerous quickly.
jose_n	. it works by tokenizing each line of code
jose_n	it then evaluates the function and then makes decisions
jose_n	based on that.
jose_n
jose_n	to deviate for a moment from the prepared text, let me expand upon this.
jose_n	czech tokenizes a line as you would do so mentally, by using the dividing components in C
jose_n	so a line like
jose_n
jose_n	printf("this is a string %s", string);
jose_n
jose_n	would be taken apart by czech:
jose_n
jose_n	printf  "this is a string %s"    string
jose_n	so it knows it is looking at a printf use, and it has two arguments
jose_n	and the first argument has a quoted use of a string (via the %s), so it calls this 'safe'.
jose_n	similarily, when you have a strcpy from a constant source string:
jose_n
jose_n	strcpy(foo, "this is foo");
jose_n
jose_n	czech sees:
jose_n	strcpy   foo    "this is foo"
jose_n	and the quotes tell it 'that is a constant source string'.
jose_n
jose_n	loop detection is done by the basic keywords 'for', 'while', 'until'. when a strcat() call is detected in there, it throws an error for you.
jose_n	this is what i mean by tokenizing the lines, and then how it evaluates the tokens.
jose_n	it also ignores keywords outside of functions, and attempts to ignore them inside of strings. so you can have a string like "i gets printf" and it will try and ignore those.
jose_n	ok. back to the prepared talk. :) [ thank you vizard for #redes!!!]
jose_n
jose_n	[page 15]
jose_n	Czech's performance is pretty amazing, given the entangled
jose_n	nature of the code. In single pass mode (as czech -C *.c) it can
jose_n	do about 1 million lines per minute. When you do full analysis
jose_n	using preprocessing, you see about a 10x loss in performance.
jose_n
jose_n	I should note that these numbers are on my laptop, a P3/600
jose_n	OpenBSD system. Filesystems, memory, and CPU will, of
jose_n	course, vary these numbers.
jose_n	Also, I have obtained these numbers by running czech against a lot of source code, like all of the OpenBSD src tree, Apache, OpenLDAP, and BIND 9.
jose_n	(yes, i turned up some notable errors)
jose_n
jose_n	[page 16]
jose_n	Here's an example of czech in action. In this trimmed example,
jose_n	czech was used in the Makefile as a replacement for CC:
jose_n
jose_n	$ make CC=czech
jose_n	czech  -O2   -c www6to4.c
jose_n	initializing czech 0130-04082002 in /home/jose/software/www6to4-1.5
jose_n	scanning www6to4.c ...
jose_n	line 120: possible buffer overflow in strcpy
jose_n	line 487: possible buffer overflow in strcpy
jose_n	line 505: possible buffer overflow in strcpy
jose_n
jose_n	total number of lines: 683
jose_n	total number of matches: 3
jose_n	And it finds a few keywords it warns you about. In the next slide
jose_n	we analyze part of the output.
jose_n
jose_n	[for those of you looking for the slides, they are on my website at
jose_n	http://www.monkey.org/~jose/presentations/czech-rubicon02.d/ ]
jose_n
jose_n	[page 17]
jose_n	Here's a brief analysis of one of the strcpy() calls czech warned
jose_n	you about:
jose_n
jose_n	481             while (fgets (buf, sizeof (buf), configfp)) {
jose_n	482                 char cmd[BUFSIZ], arg[BUFSIZ], tmp[BUFSIZ];
jose_n	483                     char *p, *q;
jose_n	484
jose_n	485                 line_num++;
jose_n	486
jose_n	487                 strcpy (tmp, buf);
jose_n
jose_n	In line 481 buf is a usersupplied vaariable, bounds checked by
jose_n	fgets(). In line 487 its called in strcpy() from buf to tmp. The
jose_n	risk here depends on the relative sizes of the buffers, tmp and
jose_n	buf. One of the things czech has to learn is how to calculate
jose_n	those values ...
jose_n
jose_n	In this case it is not an off by one error, as fgets() will accept
jose_n	sizeof(buf) - 1 characters. Both buf and tmp are defined as BUFSIZE
jose_n	in length, but the strcpy on line 487 will copy exactly BUFSIZE from
jose_n	buf to tmp with a NULL terminator.
jose_n	(with fgets() supplying the NULL termination to buf)
jose_n
jose_n	[page 18]
jose_n	So now let's talk about the limitations of czech. Like all lexical
jose_n	analysis tools for static analysis, it doesn't really get to learn
jose_n	program flow. It only looks top down in any one file, and has only
jose_n	the most basic understandings of input and output.
jose_n
jose_n	Czech cannot examine all of the possible states of the code,
jose_n	instead it takes a basic case analysis. Perhaps it should take a
jose_n	worst case analysis. Like I said above, czech doesn't calculate
jose_n	the size of a variable's allocated buffer (but it should be easy to
jose_n	do so), or the return sizes of functions. And of course it has
jose_n	some serious bugs in it which prevent me from rolling up a 0.1
jose_n	release (but you can check it out in CVS).
jose_n
jose_n	[page 19]
jose_n	I would like to finish czech, but I will have to move to yacc
jose_n	grammar to do so, I think. This will certainly make the lex code
jose_n	easier to understand. I will also have to construct a rudamentary
jose_n	C parser, and of course examine the sizes of the arguments to
jose_n	the functions for comparison. Lclint does this for example, with
jose_n	a basic prototype of:
jose_n
jose_n	strcpy(dest, src);
jose_n	/* insist (size(src) =< sizeof(dest)) */
jose_n
jose_n	I also would like it to track dangerous functions and then the use
jose_n	of those. This would allow for the tracking of seemingly innocent
jose_n	calls to be less obscure in user defined functions.
jose_n	This means you have a function foo(char arg1, int arg2) that does something dangerous.
jose_n	Let's say it opens a file in a not so safe manner.
jose_n	You mark this when you see foo() defined, but not when foo() is called
jose_n	within the rest of the source code.
jose_n	It should be enough to simply say "hey, you called foo() at line N in file.c"
jose_n	rather than trying to have czech learn why foo() is bad.
jose_n	The second generation tool is in the design phase. Its named
jose_n	'prauge', a play on the capital of the Czech republic, as well as
jose_n	the shorted "prog". I'm shooting for a dynamic analysis tool,
jose_n	possibly using the ptrace(2) functions to monitor arguments and
jose_n	how they are called, or perhaps the Gdb analysis engine
jose_n	(suggested by a couple of friends). I may be able to simply do
jose_n	static analysis from the output of the ktrace facility.
jose_n
jose_n	[page 20]
jose_n	To conclude, while it certainly has some limitations, source code
jose_n	analysis using lexical analysis techniques is worthwhile for
jose_n	development. However, it can only assist the developer, not
jose_n	replace a manual audit. Lastly, such tools are limited in the
jose_n	scope of their known grammar. Czech, an implementation of
jose_n	such a tool, was quick to code for me, and is pretty easy to use
jose_n	for most people, and runs very very fast on lots of code.
jose_n	However, its has a way to go before it can return really useful
jose_n	data in a majority of the situations.
jose_n	[page 21]
jose_n	Here are some resources for you if you would like to look at this
jose_n	information more. Czech is part of Project Pedantic, also by me
jose_n	at Crimelabs, and is available on SourceForge. The Sardonix project,
jose_n	which 'sarnold' is involved in, has a number of excellent links
jose_n	and resources. And lastly, some of the more academic aspects of
jose_n	source code analysis can be found through the Open Source Quality
jose_n	Project website. I reccomend you look at this one, especially.
jose_n
jose_n	Thank you, and I want to thank sarnold, viZard, MJesus,
jose_n	Fernand0, and Oroz, as well as all of the other people involved in
jose_n	Uninet Infosec. It's an honor to present here with such
jose_n	esteemed fellow presenters. Thank you! Also, thank you to David
jose_n	Wheeler, Rick Wash, and Tycho for great discussions on source code
jose_n	analysis, and Georgi for some impromptu spell checking.
jose_n	[done]
MJesus	clap clap clap clap clap clap clap clap clap clap
MJesus	clap clap clap clap clap clap clap clap clap clap
jose_n
MJesus	clap clap clap clap clap clap clap clap clap clap
MJesus	clap clap clap clap clap clap clap clap clap clap
viZard	PLAS PLAS PLAS PLAS PLAS PLA SPLAS PLAS PLAS PLAS PLAS
viZard	PLAS PLAS PLAS PLAS PLAS PLA SPLAS PLAS PLAS PLAS PLAS
sarnold	clap clap clap clap clap
sarnold	clap clap clap clap clap
sarnold	clap clap clap clap clap
sarnold	clap clap clap clap clap
jose_n	i would like to now open the floor up for questions here or in #qc
viZard	PLAS PLAS PLAS PLAS PLAS PLA SPLAS PLAS PLAS PLAS PLAS
sarnold	jose_n: thank you :)
viZard	PLAS PLAS PLAS PLAS PLAS PLA SPLAS PLAS PLAS PLAS PLAS
viZard	PLAS PLAS PLAS PLAS PLAS PLA SPLAS PLAS PLAS PLAS PLAS
jose_n	thank you :)
jose_n	thanks sarnold, this has been a great series.
MJesus	jose, thanks !!
MJesus	clap clap clap clap clap clap clap clap clap clap
MJesus	clap clap clap clap clap clap clap clap clap clap
MJesus	clap clap clap clap clap clap clap clap clap clap
MySQL	bravo!!!!!!!!!!!!!!!!!!!
jose_n	well done mjesus and fernand0 and vizard and oroz and everyone!
MySQL	bravo!!!!!!!!!!!!!!!!!!!
MySQL	bravo!!!!!!!!!!!!!!!!!!!
MySQL	plas plas plas plas
MySQL	plas plas plas plas
MySQL	plas plas plas plas
sarnold	jose_n: thank you :) It has worked out even better than I expected :)
jose_n	any questions i can answer now?
cronos	perl -e 'for (;;){ print(" clap  clap clap clap "); };'
jose_n	the code is available, by the way, at http://pedantic.sf.net/
jose_n	cvs only right now, like i said
jose_n	<viZard> jose, what kind of source code languages will Czech cover?
jose_n	C only
jose_n	its what i encounter most and its best to be focused on one language right now
jose_n	RATS can check C, C++, Perl, Python and Tcl i think
jose_n	Flawfinder does C and C++
jose_n	you can, of course, check C++, but you dont get real semantic checking
jose_n	<viZard> what about C++ ?
jose_n	"_
jose_n	err :)
jose_n	ie i didn't teach czech how to really think about objects and methods
jose_n	<grange> jose_n: can czech track off-by-one flows in a circles like for (ptr = buf; ...
jose_n	<jose_n> no ... i need to teach czech how to judge return sizes
jose_n	<jose_n> soon, i hope
dotslash	hasta la bye bye
jose_n	<cronos> hiya, do you have a mailing list for czech? and maybe you already answ. this one, but why the name "czech", thanks
jose_n	<jose_n> i have the normal project stuff at source forge
jose_n	<jose_n> so far i'm the only developer working on it
jose_n	<jose_n> so i haven't set up any mailing lists
jose_n	this includes user forums, though
jose_n	<MJesus> jose_n, why the name Czech ?
jose_n	<jose_n> czech in english sounds like "check"
jose_n	<sarnold> jose_n -- and so far, you haven't released any files, either ;(
jose_n	<jose_n> no ... like i said, a few bugs are keeping it from 0.1
jose_n	<jose_n> it sorta works, but not yet really well
jose_n	<jose_n> so no 0.1 yet
jose_n	<jose_n> hey, if you wanted to help :)
jose_n	the whole project has been really interesting. i've gotten to read some really cool papers, meet some nice people, and reallyu learn by trial and error.
jose_n	any more questions?
georgi	jose_n: what about removing sprintf and strcpy from all headers and libraries. well it shall break some apps ;)
jose_n	it would, but then again you have to use strncpy() and strlcpy() and such correctly, too
jose_n	you can overflow and such there.
jose_n	so its not so simple
grange	georgi: strcpy faster than strncpy
jose_n	georgi: right now a project i'm involved in is unbound
jose_n	we're auditing bind 9.2 for openbsd specific use
sarnold	jose_n: woah; theo would switch from his 4.9.x source?
jose_n	its not always as easy as s/strcpy/strlcpy/g and then arg2 == sizeof(arg0)
jose_n	no no
jose_n	its not an official work
jose_n	its for our dns servers we all use
sarnold	grange: I, for one, am willing to pay a little speed for software that is less buggy... :)
sarnold	jose_n: ah :)
jose_n	CPUs have gotten insanely fast, no sense not spending those extra cycles doing checks
grange	sarnold: if you can use strcpy in a safe manner there's no need to lost the speed
jose_n	anyhow, you have to look at the sizes of src and dest strings and if you need to NULL terminate
sarnold	grange: I think we've learned that few people can use strcpy safely :(
jose_n	grange: you can't be sure of all of the sources of the data in any function
jose_n	you should always bound check, just because you can
jose_n	similarily sprintf() -> snprintf()
jose_n	race conditions (fopen() mnisuse)
jose_n	err misuse
jose_n	my next talk, in about an hour, is on openbsd
bighawk	i dont agree with people who pertinently say that you should use strncpy and the like instead of the "insecure" counterparts
jose_n	i'll discuss the auditing strategy
jose_n	why, bighawk?
georgi	jose_n: what if arg0 is pointer? sizeof(arg0)==4 or i am missing something ?
jose_n	right, thats why you have to know the intended sizes
jose_n	so its not a simple replacement, you still have to understand where stuff is going
jose_n	thats exactly like a bug i found in map-tools in Net|OpenBSD
jose_n	err map-mbone
bighawk	it's simply a case of knowing what to do
bighawk	using strncpy() will not guarantee safe code
jose_n	just a few weeks ago (gathering data for this presentation)
jose_n	no, but its a step closer, bighawk
jose_n	and thats the key
bighawk	i think it's perhaps a start up for beginners who easily make such errors
jose_n	you cannot possibly know all possible sources of the data in any function
bighawk	and are not fully aware of it
jose_n	so beginners vs experts
jose_n	you have to be careful no matter what
grange	sometimes using strncpy without full understanding how it works is worse than using strcpy ;-)
jose_n	well, sure
bighawk	if a data source is fixed, then there's no point in not using strcpy() if you know what's going on
jose_n	right, but i would argue for consistency's sake to always bound check
bighawk	perhaps, but you wont get many "experts" to do that
bighawk	maybe it's stubbornity or not
jose_n	i dunno. sift through the openbsd src code sometime
jose_n	you'll uncommonly find strcpy() even with a static source string
grange	strncpy is not the only way to check bounds, sometimes it's enough to clearly understand how your code works
bighawk	yeah, it's almost pathetic
jose_n	fixing bad coding style is an active approach
bighawk	but i'm not a particular fan of openbsd anyway :)
bighawk	the discussion was nice though
jose_n	so the bug still exists in netbsd, i think
bighawk	well
jose_n	i sent them the patch but they haven't yet incorporated it. openbsd fixed it a day and half later
bighawk	i simply dont agree that correct usage of strcpy, strcat whatever
bighawk	should be marked bad coding style, it simply isn't.
bighawk	and it's what grange says
jose_n	if you define bad coding style as not bound checking your strings, strcpy() and strcat() qualify as a bad coding style
bighawk	people who forcibly use strncpy, etc.. not knowing what the _real_ issue is
jose_n	well, sure
bighawk	can even cause more damage in particular situations
jose_n	thats what i said above
jose_n	you can't drop it in so readily
jose_n	you have to know whats up with the source and dest strings
bighawk	but, what's the point of using strncpy() on a buffer, where it's 100% guaranteed
bighawk	that the data is static, or that the dest buffer is large enough
grange	bighawk: just paranoia ;-)
bighawk	yeh
grange	nothing more ;-)
jose_n	like i'll show you in my next talk, paranoia pays off
bighawk	well
bighawk	you know
bighawk	i honestly dont trust openbsd
jose_n	why not?
sarnold	bighawk: why not?
bighawk	they are very fixed on security
sarnold	bighawk: when formatr string bugs became known, theo spent one week tracking down nearly a thousand of the things
bighawk	so much even, that they will probably not refrain from silently fixing issues
bighawk	they find
sarnold	bighawk: whereas, linux vendors are _still_ finding those things
jose_n	not true, bighawk
bighawk	further, the attitude
bighawk	of some people there
sarnold	bighawk: btw, the CVS commit logs are publicly available
jose_n	bighawk how much interaction have you had with openbsd's core group?
bighawk	i've had some sad experiences, i'd rather stick with netbsd or freebsd
bighawk	jose_n: not much
grange	security through obscurity ;-)
jose_n	well, i quite disagree
jose_n	i've had very pleasant experiences with the net and free core teams
bighawk	i've no personal impleasant experiences with openbsd team
bighawk	but some good, trustworthy, friends have..
jose_n	far more so than with linux vendors
bighawk	another thing with openbsd is
bighawk	that it might give you a false sense of security
jose_n	i will certainly not drny that theo and art have ... well, strong personalities
sarnold	hehehe, strong personalities... :)
grange	bighawk: nobody forbid you make your own audit ;-)
jose_n	however
bighawk	grange: of course not, it's not about that
jose_n	you have the code
jose_n	you see what they do
jose_n	you are free to make changes
jose_n	(and i certainly do)
bighawk	yes, but same thing for netbsd/freebsd
bighawk	take netbsd
bighawk	netbsd makes no such claims
jose_n	*shrug*
bighawk	but it's very accurate with security
grange	bighawk: openbsd's team really do a lot in security improvement
jose_n	the best thing you can do is use a system you know well
bighawk	that's one of the things i dislike about openbsd, when you get on the frontpage "3 years without a hole!"
jose_n	cuz then you know how to secure it
jose_n	cuz its true?
grange	e.g. recent signals audit
jose_n	four years without a remote hole in the default install
cronos	3 years without a "remote" hole (i guess they mean)
bighawk	jose_n: it might be strictly true, but they are referring to a default install
jose_n	thats what they say
jose_n	thats what is true
bighawk	which is very stripped down
jose_n	dude, portmap is open on the default install
bighawk	and?
jose_n	thats not stripped down
sarnold	yeah, if portmap is open, and they haven't had holes in it ... amazing :)
cronos	agrred
jose_n	however, openbsd never claims to be perfect
bighawk	well, openbsd default install has nearly no running network services
jose_n	the mail(1) local root hole was utterly stupid
bighawk	yes
bighawk	anyway, to get back to the false sense of security
jose_n	bighawk: so its not running ftpd, telnetd, sshd, smtp, portmap, bind, ldap, nntpd, etc ...
jose_n	you call that stripped down?
cronos	s/agrred/agreed
grange	nice discussion ;-) everybody forgot about czech ;-)
bighawk	no, but many of those services
bighawk	will be enabled for an internet server
jose_n	you're perfectly free to turn them on
bighawk	and let's take, ftpd for example
sarnold	indeed
jose_n	right: audited nice.
bighawk	there were issues with ftpd
jose_n	sure
jose_n	everyone had issues
jose_n	like i said, they never claim perfection
jose_n	they're just making a marketing claim that is true
jose_n	and demonstratable
bighawk	anyway, i think that naive people running openbsd
bighawk	will get the feeling that they are secure
bighawk	and will nto watch their other good security practice
sarnold	i just don't get that
bighawk	of course thare are arguments against that
sarnold	given the choice of running a system with audited code, and unaudited code, I'll take audited code any day
jose_n	demonstrate another general purpose os that has a) the track record of as few security issues as openbsd and b) is as vigilant about them
bighawk	jose_n: note a defualt install
cronos	bighawk there are a few of choices in *bsd and linux, is best to use the ones we feel more confortable with, is easy part is complaining about it, the hard part is fixing the problems yourself, as jose said, you have the option of fixing the openbsd source code to fit your needs, even if the openbsd team does not like/accept your patch
bighawk	almost no system will keep a default install
jose_n	that doesn't mean they've left the rest of the code unaudited
bighawk	i'm not saying that
jose_n	you're not making a very clear point, or a very strong one
jose_n	i'm not denying everything you
jose_n	re saying
jose_n	however i think you're making mountains out of molehills and seeing things
grange	bighawk: what's your favorite OS? ;-)
bighawk	it's more the general attitude around it i dislike
Lulusita	I'am cuban, hello!!!!!!!!!!
jose_n	hola lulusita
jose_n	attitudes abound
jose_n	bighawk
bighawk	and there really are bad experiences ->
sarnold	Lulusita: perhaps you are looking for #latertulia
bighawk	people getting packeted in #openbsd@efnet after having critique on something
jose_n	ok, so some users are stupid
jose_n	so what
sarnold	bighawk: nothing says you must hang out with the users of the OS :)
bighawk	people getting offensive mail
jose_n	they're not openbsd core people
bighawk	when mailing the openbsd mailing list
jose_n	arts not offensive, he's right
bighawk	asking them about some issue
bighawk	offensive as in
jose_n	theo isn't offensive unless you're repeatedly wrong
bighawk	"we're not vulnerable you idiot"
jose_n	so he doesn't have velvet gloves
bighawk	or "there's no exploit avaialbe yet, retard"
bighawk	especially the last
bighawk	scared me.
jose_n	*shrug*
jose_n	both theo and art can be .. gruff
jose_n	no one is denying that
cronos	bighawk we are suppoced to be taking about czech here now
bighawk	i dont know who mailed that back
jose_n	they are, however, right for more often than they are wrong
bighawk	cronos: oh, excuse me
cronos	np, is just a reminder
Lulusita	hello!!!!!!!!!!!
jose_n	thanks cronos
jose_n	even if you dont like some parts of openbsd, i want to remind you that evereyone can learn a lot from it
sarnold	Lulusita: perhaps you are looking for #latertulia
jose_n	and all code and actions on the code are public
cronos	Lulusita we are having a presentation here now
bighawk	jose_n not the case with free/net bsd?
jose_n	unlike, say, the linux kernel, where alan wont discuss change logs
jose_n	no, its the same there, i'm not denying that
jose_n	i'm not dissing them at all
jose_n	they're great products
sarnold	bighawk: did netbsd have thousands of format bug fixes checked in within the first week of their public knowledge?
jose_n	i use net and free, too
Lulusita	Do you speak spanish?
bighawk	sarnold: i would have no idea, and what's the appropriateness of that question?
sarnold	bighawk: netbsd must be good .. a lot of people use it.
cronos	Lulusita me parece que lo que buscas esta en #latertulia
sarnold	bighawk: however, OPenBSD had thousands of format bugs fixed in the first week of their public knowledge.
cronos	o well : )
sarnold	bighawk: you just don't get that level of security awareness _anywhere_ else.
bighawk	maybe it's also, that i generally dont like the follow 'hypes'
jose_n	openbsd aint no hype
bighawk	i think it is
jose_n	why?
jose_n	its been around for 6.5 years
bighawk	yes
jose_n	i think you're being too closed minded here
jose_n	all cuz theo called someone a retard. big shock!
bighawk	maybe
bighawk	further
bighawk	i didnt say it was theo, or whoever
jose_n	theo and art are at the top of the list of suspects :)
cronos	retard? i have been called worst things : 0
bighawk	hehe
jose_n	anyhow, all i can say is try and keep an open mind. i do
bighawk	then performance/feature wise is openbsd not close to free and netbsd
jose_n	not true
jose_n	highest packet handling performance
jose_n	define features?
bighawk	from my personal experience, freebsd performs signficantly better than openbsd
jose_n	stuff for a workstation? openbsd lags far behind free. and net.
bighawk	posix-completeness
jose_n	heh .. personal experience. tell you what, do some lab tests.
cronos	i have noticed that a lot very gifted programmers had not had time to develop personal skills, one reason could be the age, another is that they spend too much time behind the keyboard, but WTH, no one is perfect
jose_n	uh .. openbsd is the leader of the three
bighawk	for example
dotslash	Corporate deadlines .... and pressure from boss == lax code
bighawk	jose_n: smp-support, another thing, however same for netbsd
cronos	also, less time to iterate with hummans
jose_n	smp: yes. however, you have security impliecations you just cannot imagine on that.
sarnold	aye, open does lack smp support
jose_n	and that sucks
jose_n	we would love to be deploying openbsd dual cpu boxeds
sarnold	theo is working on a neato plan to use other CPUs for crypto purposes, which woudl be slick :)
jose_n	alas, we cannot
jose_n	hell yeah that would be cool!
bighawk	uhuh, heh
cronos	jose_n openbsd dual cpu boxeds? for what type of applications?
bighawk	i dont think that's where people buy multiprocessor systems for :)
jose_n	(we meaing my employer. we deploy a product based on openbsd kernels)
cronos	i see
sarnold	bighawk: true enough, but I would if I could run a stegfs on the thing :)
jose_n	bighawk: data centers, ssl processing
jose_n	thats .. well, thats a reasonable thing to do
sarnold	bighawk: or, VPN
bighawk	yes, imagine a 4-cpu board
bighawk	you would use 3 entire cpu's for cryprography acceleration?
jose_n	*sigh*
bighawk	that would probably mean 2 cpu's completely idle
cronos	i think that openwall might  a good contender of openbsd
jose_n	so do you get a kick out of misunderstanding people?
sarnold	I wouldn't buy such a board for an OS that could do only two CPUs
jose_n	wb oroz
sarnold	cronos: agreed
cronos	i personaly like to use linux when ever i can
bighawk	sarnold: yes, that's very true
jose_n	cronos: it will take a while for openwall to get up to speed
jose_n	largely cuz of the libc issues ...
jose_n	#infosec: where flamewars act as a preview of the next talk!
bighawk	even if the 2nd cpu is completely used for cryptography, i think it will be idle most of the time
cronos	i think they have a 4.4 patch, that will aallow smp
cronos	and also iptables
cronos	it all takes time
cronos	and effort
sarnold	cronos: at least the linux kernel does SMP pretty well
jose_n	not true
jose_n	http+ssl, vpns, etc ...
jose_n	all extremely heavy on operations
bighawk	yes, of course
dotslash	heh
bighawk	jose_n: but full cpu utilisation?
jose_n	yes
jose_n	at a data center, a gateway device on a good network, etc
* cronos is a linux nut : )
jose_n	openbsd isn't just for home use
bighawk	well
bighawk	at a datacenter
jose_n	now quit trolling, ok?
dotslash	does obsd run on PowerPC chips?
jose_n	yes
bighawk	it's kinda dubious if single-cpu'd openbsd can handle the load-at-all
jose_n	macppc and ppc prep boards
cronos	sarnold it also has a great community
jose_n	bighawk: you have clearly shown that you dont know what you're talking about
dotslash	ive been running linux on my ibook a while cuz osx sucks but obsd would be nice
jose_n	(openbsd smugness comes out of jose!)
sarnold	jose_n: I don't know, it worked _very_ well for me at my home :)
sarnold	dotslash: probably, it runs on damn near everything :)
bighawk	jose_n: oh, and why?
jose_n	dotslash: depends on the model
jose_n	bighawk: plz stop trolling
dotslash	ive got netbsd on my dreamcast which is ppc based
dotslash	=]
jose_n	yeah, i use netbsd on my powermac 8500 (no adb support in openbsd/macppc)
sarnold	dotslash: sadly, I couldn't figure out how to the apple titanium board to boot the CDs :(
dotslash	sarnold !
dotslash	yes thats my issue
jose_n	OF blues, sarnold?
jose_n	i haven't mucked with the new OF.
dotslash	I used some multi-boot cd or something said it should work
sarnold	OF ?
jose_n	open firmware
dotslash	I believe so
bighawk	jose_n: it's not nice to accuse people of trolling, isn't it a realistic assumption that single-cpu'd systems can handle less load than multiprocessor systems?
dotslash	open firmware
* sarnold knows next to nothing about apple hardware
sarnold	bighawk: no, not at all
bighawk	jose_n: would multi-cpu systems be placed at such places for nothing?
cronos	bighawk it just happens that jose is our invited guest here
jose_n	bighawk: thats true that 1 cpu boxes can't handle as much as 2 cpu boxes. however, the rest of your remarks are nothing but trolling at this point.
jose_n	now knock it off, plz.
cronos	and he is asking you politely
jose_n	look, i've answered your questions, i've corrected your erroroneous statements, and i've made several correct concessions.
jose_n	you should try and do the same.
dotslash	bighawk are you the man whos shellcode I see all over the place?
bighawk	i'm being polite all the time
cronos	bighawk the issues that you bring are beyond the scope of this conference
bighawk	dotslash: uhuh
jose_n	:)
bighawk	dotslash: are you the guy who needed help on string enumeration?
dotslash	I am KF on bugtraq
bighawk	that's what i'm saying...
dotslash	sure enough
jose_n	nice shellcode, bighawk
jose_n	small, efficient :)
cronos	dotslash gretings : )
jose_n	oh hey, kf, didn't recognize you
dotslash	i need to meet more ppc guys ... ppc shellcode is hard to come by
cronos	he
dotslash	hola fellas
* jose_n nods
dotslash	=]
jose_n	LSD help you out at all ./
bighawk	jose_n: well, i just want to get any misunderstandings out of the way, i did not attempt to troll you or anybody.
jose_n	they had a bunch
dotslash	yeah actually quite a bit ... its mostly AIX stuff though
jose_n	bighawk: i think i've tried to correct your misunderstandings
jose_n	ahh .. yeah, thats true
bighawk	jose_n: so sorry if it felt like that to you..
jose_n	and ppc is .. well, difficult to shellcode on;
jose_n	the registers are way cool
dotslash	yes hence why I like it
dotslash	I wanted to learn RISC and this was the cheapest way to go
cronos	jose_n psilosibie cubensis is better, i think, is also organic, so dont panic
jose_n	yeah ... that or maybe an old sparc?
dotslash	hacking up my dreamcast was not realistic
jose_n	heh ...
bighawk	dreamcast = SuperH
dotslash	sparc registers are even more fscked up
jose_n	yeah ... but far more info on them
jose_n	have you looked at the ibm and mot docs?
bighawk	SuperH is one awesome architecture
dotslash	bighawk yes dreamcast is fun to play with ... it has a nice graphics accelerator too
dotslash	good for learning to draw in 3d
bighawk	i dont give too much about 3d stuff
dotslash	jose_n yeah I have lots of paers / docs / books on the processor
jose_n	ah ok .. sorry, i dont asm, so .. never had to look
dotslash	bighawk nor I but its funn when your mucking in gdb
bighawk	dotslash: what made you request help for string enumeration? it was rather entertaining to read that.
dotslash	man I play with all kinds of weird stuff in our labs
dotslash	I think REAL randomly
dotslash	I cant think of what started it though
dotslash	heh
dotslash	I need some help with malloc on PPC like seriously
bighawk	heh?
dotslash	laughing cuz I cant remember
dotslash	=]
dotslash	ill be back in a bit also guys ...
dotslash	off to try to chop my leg off with my rc helicopter
dotslash	=]
dotslash	bighawk I remember now btw... it was for making sequences of 4 byte combinations for powerpc instructions i believe
jose_n	oh yeah. ...
dotslash	i have been trying to come up with a xor decoder for ppc cuz all my shellcode has LOTS of nulls
dotslash	so i thought I would just use only alpha numeric chars in my shellcode
sarnold	jose_n: heh, four minutes until 1900 ... you should take a break for at least two of those minutes :)
jose_n	dotslash you seen K2's stuff?
dotslash	come up with a small db of possible functions and then work from there
jose_n	heh .. ok
dotslash	yes some of it ... he did something with ADMmutate right
jose_n	yeap
dotslash	yeah i remember trying to steal the PA-RISC stuff from it
dotslash	and trying to adapt to powerpc
jose_n	wb georgi
sarnold	aw, crap
sarnold	I forgot I parked my car at a meter
sarnold	I gotta go
sarnold	i'll be back
jose_n	heh ...
cronos	georgi aloha
sarnold	(go whenever yo ure ready :)
dotslash	muchos gracias

Generated by irclog2html.pl 2.1 by Jeff Waugh - find it at freshmeat.net!